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NATIVE AMERICAN YOUTH 


Agencies Incorporated Almost All Leading Practices 
When Assessing Grant Programs That Could Prevent 
or Address Delinquency 


What GAO Found 


The Departments of Justice (DOJ), Health and Human Services (HHS), the 
Interior (Interior), and Education (Education) administered at least 38 grant 
programs from fiscal years 2015 through 2018 that could have helped prevent or 
address delinquency among Native American youth. These agencies made 
about $1.9 billion in awards to grantees through these programs during this 
period. 


These agencies incorporated almost all of the leading practices GAO identified 
for performance measurement or program evaluation when assessing the 
performance of selected grant programs. For example, HHS’s Administration for 
Children and Families (ACF) incorporated 13 of the 14 leading practices for 
performance measurement but did not fully assess grantee data reliability for one 
of its programs. By developing a process to assess the reliability of grantee data 
contained in the annual performance reports that tribal recipients submit, ACF 
could obtain further assurance that it has an accurate representation of grantee 
performance. GAO also found that Interior's Bureau of Indian Education (BIE) did 
not conduct formal data reliability checks on performance data that grantees 
report and did not always collect performance reports from grantees in a timely 
manner for one of its programs. By developing a process to assess the reliability 
of a sample of grantee performance data and taking steps to alert grantees when 
they are late in submitting performance reports, BIE could better ensure that 
grantees are complying with the terms and conditions of the grant program and 
better understand how the program and its grantees are performing. 


Officials in all 12 interviews with tribes or tribal consortia GAO interviewed cited 
risk factors that contribute to juvenile delinquency in their communities. 


Number of Interviews in Which Tribal Officials Cited Risk Factors Contributing to Juvenile 
Delinquency 
Risk factor for juvenile delinquency 


Substance misuse 


Educational challenges 


Mental health 


Interpersonal violence 
Poverty 
Historical trauma 
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Number of interviews 
Source: GAO analysis of interviews with officials from federally recognized tribes and tribal consortia. | GAO-20-600 
Note: The figure includes the most common risk factors tribal officials cited for juvenile delinquency. 
While tribal officials cited restrictions placed on federal grant funding, difficulty 
communicating with program staff, and challenges hiring and retaining staff as 
barriers to implementing federal programs, they also identified promising 


practices, such as executing culturally relevant programs, for preventing or 
addressing juvenile delinquency. 
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i U.S. GOVERNMENT ACCOUNTABILITY OFFICE 
441 G St. N.W. 
Washington, DC 20548 


August 6, 2020 


The Honorable John Hoeven 
Chairman 

Committee on Indian Affairs 
United States Senate 


The Honorable John Barrasso 
United States Senate 


American Indian and Alaska Native (Native American) youth face unique 
challenges when it comes to their contact with the justice system. 
According to federal and other reports, issues such as historical trauma, 
exposure to violence, and substance abuse, coupled with high poverty 
rates and tribal communities’ lack of funding for mental health and other 
services, may make some Native American youth susceptible to 
becoming involved with the justice system.’ These reports also note that 
tribal justice systems may lack resources when compared with federal 
and state justice systems, which themselves may not have culturally 
appropriate programming to meet the needs of Native American youth. 


Federal agencies, including the Departments of Justice (DOJ), Health and 
Human Services (HHS), the Interior (Interior), and Education (Education), 
provide financial assistance that could potentially help prevent or address 
delinquency among Native American youth.2 In 2017, however, the 
Council of the Inspectors General on Integrity and Efficiency issued a 
report that identified common vulnerabilities in federal programs serving 
Native American communities across several agencies’ Office of the 
Inspector General audits, such as deficiencies in assessing the quality 
and effectiveness of federal programs benefiting Native American 


1For example, see the Attorney General’s Advisory Committee on American Indian/Alaska 
Native Children Exposed to Violence, Ending Violence so Children Can Thrive 
(Washington, D.C.: November 2014); and Indian Law and Order Commission, A Roadmap 
for Making Native America Safer: Report to the President and Congress of the United 
States (November 2013). 


2According to the DOJ Office of Juvenile Justice and Delinquency Prevention (OJJDP), a 
delinquent act is an act committed by a juvenile for which an adult could be prosecuted in 
a criminal court but, when committed by a juvenile, is within the jurisdiction of the juvenile 
court. Delinquent acts include crimes against persons, crimes against property, drug 
offenses, and crimes against public order, when juveniles commit such acts. 
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communities.3 In the same year, we added to our High Risk List Interior 
and HHS’s management of education, energy, and health care programs 
that serve tribes and their members, given concerns about the overall 
performance of their federal programs that serve Native American 
populations. 4 


In September 2018, we reported on what available data show about the 
number and characteristics of Native American youth in the federal, state, 
local, and tribal justice systems. We found that the number of Native 
American youth in federal as well as state and local justice systems 
declined across the arrest, adjudication, and confinement phases of the 
justice system from 2010 through 2016.5 We also reported on 
discretionary grants and cooperative agreements that federal agencies 
awarded from fiscal years 2015 through 2017 that could have helped 
prevent or address delinquency among Native American youth. We 
identified at least 122 DOJ and HHS discretionary grants and cooperative 
agreements across several issue areas, such as violence or trauma, 
justice system reform, suicide prevention, and alcohol and substance 
abuse. We also found that tribal governments and Native American 


3Council of the Inspectors General on Integrity and Efficiency, Vulnerabilities and 
Resulting Breakdowns: A Review of Audits, Evaluations, and Investigations Focused on 
Services and Funding for American Indians and Alaska Natives (Washington, D.C.: 
December 2017). 


4In 2017, we identified weaknesses in Interior's Office of the Assistant Secretary for Indian 
Affairs’s oversight of personnel responsible for inspecting Bureau of Indian Education 
(BIE) school facilities for safety, and management of BIE school construction projects. We 
also reported that the Office of the Assistant Secretary for Indian Affairs’s lack of oversight 
of BIE school safety contributed to deteriorating facilities and equipment in school 
facilities. See GAO, High Risk Series: Progress on Many High-Risk Areas, While 
Substantial Efforts Needed on Others, GAO-17-317 (Washington, D.C.: Feb. 15, 2017). In 
March 2019, we reported that, as of December 2018, the Office of the Assistant Secretary 
for Indian Affairs had made progress in addressing some areas of our recommendations 
to address these deficiencies, such as taking actions to monitor corrective measures that 
address weaknesses with the agency’s safety program—which covers safety at BIE 
schools. However, challenges remain, including that the agency has not yet demonstrated 
that it is monitoring whether relevant employees are being held to the agency’s required 
performance standards for safety inspections. We also found that the Office of the 
Assistant Secretary for Indian Affairs has not put in place a comprehensive, long-term 
capital asset plan to inform allocation of school facility funds. See High Risk Series: 
Substantial Efforts Needed to Achieve Greater Progress on High-Risk Areas, 
GAO-19-157SP (Washington, D.C.: Mar. 6, 2019). 


5We generally analyzed calendar year data at the state and local levels and fiscal year 
data at the federal level. See GAO, Native American Youth: Involvement in Justice 
Systems and Information on Grants to Help Address Juvenile Delinquency, GAO-18-591 
(Washington, D.C.: Sept. 5, 2018). 
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organizations applied primarily for discretionary grants and cooperative 
agreements that specified tribes or Native Americans as a primary 
beneficiary. 


You asked us to review information about Native American youth in the 
justice system. This report addresses (1) the extent to which federal 
agencies provide financial assistance targeted to tribes that could help 
prevent or address delinquency among Native American youth; (2) the 
extent to which federal agencies assess the performance of selected 
grant programs that could help prevent or address delinquency among 
Native American youth and the extent to which they incorporate leading 
practices when doing so; and (3) juvenile delinquency issues tribes report 
encountering within their communities and the challenges they face in 
preventing or addressing these issues. 


To address our first objective, we identified relevant federal financial 
assistance available from fiscal years 2015 through 2018, which included 
discretionary grants, cooperative agreements, and formula grants 
(collectively referred to as grant programs throughout the report).6 To 
identify relevant grant programs, we searched Grants.gov—an online 
repository that houses information on over 1,000 different programs 
across federal agencies—as well as agencies’ websites.” We then 
selected discretionary grant programs whose summary descriptions or 
purposes explicitly mentioned tribes or Native Americans as 
beneficiaries’ and whose activities related to the risk or protective factors 


6We selected this period to focus on the most recent trends in available funding from 
when we began the review. According to GAO-11-646SP, a program may be any activity, 
project, function, or policy that has an identifiable purpose or set of objectives. See GAO, 
Performance Measurement and Evaluation: Definitions and Relationships, 
GAO-11-646SP (Washington, D.C.: May 2, 2011). 


7Within Grants.gov, we specifically searched for grant programs within nine federal 
agencies because they accounted for about 97 percent of federal funding that served 
Native American communities in fiscal year 2018. These federal agencies included DOJ, 
HHS, Interior, and Education, among others. For example, according to a fiscal year 2019 
Federal Funding for Program Serving Tribes and Native American Communities budget 
document, federal agencies enacted approximately $22.017 billion for programs that 
served Native Americans in fiscal year 2018, and approximately $21.431 billion, or about 
97 percent, came from nine federal agencies. 


8We excluded grant programs that focused on providing services for victims of crimes and 
violence from this review just as we did in GAO-18-591. We also excluded selected 
Interior’s National Park Service discretionary cooperative agreements that facilitated 
Native American youth involvement in tribal cultural resource projects but did not accept 
applications from other entities. 
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discussed in the Office of Juvenile Justice and Delinquency Prevention’s 
(OJJDP) Tribal Youth in the Juvenile Justice System literature review, 
among other criteria.2 For formula grant programs, we also selected as 
relevant those grant programs whose activities related to the risk or 
protective factors discussed in the aforementioned literature review as 
well as those for which tribal entities were eligible to apply directly for 
funding. After developing an initial list of relevant grant programs based 
on our review of Grants.gov search results and agency websites, we 
worked with officials from HHS, DOJ, Interior, and Education—the 
agencies from which we initially identified relevant programs—to identify 
any additional relevant programs. Despite these steps, it is possible that 
we did not identify all potentially relevant federal grant programs targeted 
to tribes that could prevent or address delinquency among Native 
American youth. Moreover, for some grant programs we identified, 
addressing juvenile delinquency is not the primary purpose and 
beneficiaries may include youth who are neither Native American nor 
have a history of delinquency. We included them as programs that could 
address juvenile delinquency because, as mentioned above, their 
described activities relate to the risk or protective factors discussed in the 
OJJDP Tribal Youth in the Juvenile Justice System literature review, 
among other criteria. 


After finalizing the list of relevant grant programs, we obtained award data 
for these programs for fiscal years 2015 through 2018 from each of the 
agencies. We assessed the reliability of the award data by reviewing 
related documentation, performing checks for missing data and other 
obvious errors, and asking knowledgeable officials questions about their 
agencies’ information systems that contain these award data as well as 
how they use these systems. We determined that the award data were 
sufficiently reliable for the purposes of reporting funding trends for grant 


90 ffice of Juvenile Justice and Delinquency Prevention, Literature Review: A Product of 
the Model Programs Guide—Tribal Youth in the Juvenile Justice System (Washington, 
D.C.: last updated April 2016). This literature review identified risk factors for tribal youth, 
which are characteristics or activities that could contribute to a higher likelihood of their 
contact with the criminal justice system. These risk factors included historical trauma, 
violence, suicide, substance use, and lack of cultural instruction. The literature review also 
listed various protective factors—characteristics of the child, family, and wider 
environment that can increase resiliency and reduce the likelihood of negative child 
outcomes and behaviors, such as contact with the juvenile justice system. The protective 
factors in the literature review included family and culture. 
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programs that could have helped prevent or address delinquency among 
Native American youth from fiscal years 2015 through 2018. 


In addition to identifying relevant grant programs, we also interviewed 
agency officials who manage self-determination contracts and self- 
governance compacts to determine the extent to which tribes might use 
financial assistance from these sources to prevent or address 
delinquency among Native American youth. 1° 


For our second objective, we conducted interviews with or submitted 
questions for written responses to officials from HHS, DOJ, Education, 
and Interior to determine the extent to which agencies assessed the 
performance of selected grant programs.'! We also selected a 
nonprobability sample of 15 grant programs that these agencies 
administered from the relevant programs we identified from fiscal years 
2015 through 2018 that we identified as relevant for the first objective. In 
selecting this sample, we employed a multistage process that considered 
(1) the federal agency that administers the program, (2) the agency 
component office that manages the program, and (3) the program type 
(that is discretionary versus formula grant program). '2 In considering 
these three factors, we were able to select a sample that represented a 
mix of agencies, component offices, and types of programs. For these 
programs, we reviewed documentation such as performance reports for 
one to two grantees of each of the 15 programs. 


10Authorized by the Indian Self-Determination and Education Assistance Act of 1975, as 
amended, self-determination contracts and self-governance compacts are agreements 
between tribes (or tribal organizations) and Interior and HHS’s Indian Health Service. Self- 
determination contracts and self-governance compacts are agreements that enable tribes 
to assume the administration of certain federal programs—or portions of them—that the 
agencies themselves previously managed. See Pub. L. No. 93-638, 88 Stat. 2203 
(classified as amended at 25 U.S.C. §§ 5301-10); see also 25 U.S.C. § 5304(I). We 
discuss these agreements in more detail in later sections of this report. 


11For the purposes of this report, assessing performance refers to agencies determining 
the extent to which a grantee or grant program has made progress on its goals or how 
well a grant program is working as measured by the incorporation of leading practices we 
identified for performance measurement and program evaluation. 


12We did not include self-determination contracts and self-governance compacts tribes 
have with HHS’s Indian Health Service and Interior. This is because monitoring of the 
implementation of these agreements is governed by the Indian Self-Determination and 
Education Assistance Act, as amended, and its implementing regulations, and monitoring 
and performance measurement standards that generally apply to federal grant programs 
do not generally apply to these agreements. 
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In addition, we determined the extent to which HHS, DOJ, Education, and 
Interior incorporated leading practices for performance measurement or 
program evaluation when assessing the performance of their grant 
programs.’ To do this, we (1) identified leading practices for performance 
measurement and program evaluation and (2) determined the extent to 
which agencies incorporated these leading practices for the 
nonprobability sample of 15 programs we previously mentioned. We 
identified leading practices for performance measurement and program 
evaluation by analyzing the content of relevant GAO and external 
guidance and reports to identify initial leading practices.14 We then 
combined those initial leading practices that we determined were 
substantially similar. We finalized the list of leading practices based on 
feedback from internal specialists. In total, we identified 14 leading 
practices for performance measurement and 14 leading practices for 
program evaluation. See appendix | for the list of leading practices we 
identified for performance measurement and program evaluation and 
appendix II for the list of sources we reviewed to identify them. 


We then assessed the extent to which agencies incorporated the leading 
practices we identified by conducting interviews with or submitting 
questions for written responses to cognizant agency officials. We also 
analyzed program documentation for the grantees of the nonprobability 
sample of 15 grant programs as well as program evaluation reports 
published between fiscal years 2015 and 2018 that focused on Native 
American youth regardless of program.‘ Despite these steps, it is 
possible that we did not identify every program evaluation report that 


13See GAO, Best Practices Methodologies: A New Approach for Improving Government 
Operations, NSIAD-95-154 (Washington, D.C.: May 1, 1995). According to this report, 
best management practices refer to the processes, practices, and systems that are widely 
recognized as improving an organization’s performance and efficiency in specific areas. 
For the purpose of this review, the term “leading practices” refers to commonly mentioned 
and recommended practices related to performance measurement and program 
evaluation that we identified when reviewing GAO and external reports or guidance 
relevant to these topics. 


14Among others, we reviewed GAO reports such as Designing Evaluations: 2012 
Revision, GAO-12-208G (Washington, D.C.: Feb. 1, 2012) and external guidance such as 
the American Evaluation Association’s Guiding Principles for Evaluators (Washington, 
D.C.: July 2004). 


15We did not assess the extent to which an agency or component office incorporated 
leading practices for program evaluation if we could not identify an evaluation report they 
published from fiscal years 2015 through 2018 that focused on Native American youth. 
We also did not assess reports that, in our judgement, did not meet the definition of 
program evaluation contained within GAO-11-646SP. 
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focused on Native American youth published during this period. 


Additionally, because we relied on limited nonprobability samples of grant 
programs, grantees, and program evaluation reports to determine the 
extent to which agencies incorporated the leading practices we identified, 
our findings are not generalizable to the universe of federal programs, 
grantees, and program evaluation reports. However, they do offer insights 
into agencies’ performance measurement and program evaluation 
practices, particularly as those practices relate to grant programs targeted 
to tribal communities. 16 


To address our third objective, we conducted 12 semistructured 
interviews with officials from 10 federally recognized Indian tribes (tribes) 
and two tribal consortia. We selected these interviewees to include tribes 
or tribal consortia located near our site visit locations but also considered 
geographical diversity as well as a mix of smaller and larger tribes.17 We 
then identified common trends across the interviews we held using NVivo, 
a qualitative analysis software program. '8 In addition to these interviews, 
we also conducted two discussion groups at tribal-focused conferences 
with about 30 officials who volunteered to participate. The perspectives 
that the tribes, tribal consortia, and the two discussion groups offered are 
not generalizable, but they provide insights into the juvenile delinquency 
issues that some tribes encounter within their communities, the types of 
challenges they face in preventing or addressing these issues, and the 
practices for addressing these challenges that they consider promising. 
For further information on our scope and methodology, see appendix I. 


We conducted this performance audit from August 2018 to August 2020 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 


16Although we evaluated the extent to which agencies incorporated leading practices for 
performance measurement and program evaluation, we did not assess the performance of 
specific grant programs themselves. 


17We conducted site visits with federally recognized tribes and agency officials in 
California and New Mexico. 


18We counted an interview as mentioning a specific issue or challenge related to 
preventing or addressing juvenile delinquency if at least one of the tribal officials 
participating in the interviews discussed it. 
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the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


SS 


Background 


Native American 
Population and Indian 
Land Areas 


Approximately 6.9 million people in the United States identified as Native 
American based on 2018 U.S. Census annual population estimates, of 
which about 29 percent were under 18 years old.19 As of January 2020, 
the federal government has recognized 574 Indian tribes, thereby making 
them eligible to receive funding and services from Interior.2° According to 
Interior's Bureau of Indian Affairs, as of June 2018, there were 
approximately 497 Indian land areas in the United States administered as 
federal Indian reservations or other tribal lands (for example, pueblos, 
villages, and communities). These land areas, which span more than 56 
million acres and 37 states and vary in size, can generally be referred to 
as Indian country.21 Indian country is often in remote, rural locations but 
also can be found near urban areas. Native Americans live both inside 
and outside of these land areas, and people who live in Indian country 
may include those who identify as Native American and those who do not. 
Figure 1 illustrates areas in the United States with relatively high 
concentrations of Native Americans. 


18The U.S. Census Bureau defines American Indian or Alaska Native as “A person having 
origins in any of the original peoples of North and South America (including Central 
America) and who maintains tribal affiliation or community attachment.” The U.S. Census 
Bureau adheres to the 1997 Office of Management and Budget standards on race and 
ethnicity, which guide the Census Bureau in classifying written responses to the race 
question. The 1997 Office of Management and Budget standards permit the reporting of 
more than one race, and an individual’s response to the race question is based upon self- 
identification. For the purposes of this report, we use the term “Native American” to refer 
to American Indian or Alaska Native individuals, as discussed above. 


20See Indian Entities Recognized by and Eligible to Receive Services from the Bureau of 
Indian Affairs, 85 Fed. Reg. 5462 (Washington, D.C.: Jan. 30, 2020). 


21See 18 U.S.C. § 1151. With certain exceptions, there is generally no Indian country in 
Alaska. As an example of size variation, the Navajo Nation spans New Mexico, Arizona, 
and Utah and consists of approximately 27,000 square miles, whereas certain areas of 
Indian country in California consist of less than 1 square mile. 
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Figure 1: Map of Native Americans in the United States as a Percentage of County Population, according to the 2014-2018 


American Community Survey 


Source: United States Census Bureau. | GAO-20-600 


Federal Financial 
Assistance That Could 
Help Prevent or Address 
Juvenile Delinquency in 
Tribal Communities 


Percentage of Native American 
of county total 


= Less than 1.5 

[ 1.5 to less than 3 
Lf] 3 to less than 8 
[| 8 percent or more 


Federal agencies, including HHS, DOJ, Education, and Interior, provide 
financial assistance that could potentially help prevent or address juvenile 
delinquency in tribal communities. This financial assistance could include 
discretionary grants, cooperative agreements, and formula grants, as well 
as self-determination contracts and self-governance compacts. 


Discretionary grants. Discretionary grants are competitive in nature, 
whereby the granting agency has discretion to choose one applicant over 
another based on eligibility and merit. 


Cooperative agreements. Cooperative agreements are similar to 
discretionary grants in that federal agencies generally award them to 
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grantees competitively based on eligibility and merit. In contrast to 
discretionary grants, federal agencies generally use cooperative 
agreements when they anticipate that there will be substantial federal 
involvement with the recipient during the performance of the program 
activities. 22 


Formula grants. Formula grants are a type of noncompetitive grant that 
agencies award to grantees based on statistical criteria. Authorizing 
legislation and regulations usually define the statistical criteria and the 
funding amount allocated to recipients of formula grants. Although federal 
agencies generally award formula grants to state, territorial, and local 
governments, tribes can also receive funding from these grants. Tribes 
may receive these grants either through pass-through awards that allow 
entities such as state governments to make subawards to tribes to carry 
out the public purpose of the program within their jurisdiction or by 
applying directly to receive funding, if they are eligible. 28 


Self-determination contracts and self-governance compacts. Self- 
determination contracts and self-governance compacts are agreements 
that authorize tribes or tribal organizations to assume administration of 
certain federal programs—or portions thereof—that the HHS’s Indian 
Health Service or Interior previously managed.24 These agreements are 
not grant programs. Although both types of agreements transfer the 
administration of federal programs to tribes or tribal organizations, there 
are differences between the agreements. For example, self-governance 
compacts generally allow tribes and tribal organizations to redesign or 
consolidate programs pursuant to the terms of the compact. In contrast, 
tribes and tribal organizations that have entered into self-determination 
contracts with HHS’s Indian Health Service or Interior would need to 


22See 31 U.S.C. § 6305; see also 59 Comp. Gen. 758 (1980) (stating that “the only basic 
distinguishing factor between grants and cooperative agreements under the statute is the 
degree of federal participation during performance.”). 


23Federal agencies also award another type of grant primarily to government entities— 
block grants. According to Grants.gov, a block grant is funding assistance that the federal 
government often awards to state or territorial governments that generally allows for 
autonomy and flexibility to the grantees to decide how to implement the program as long 
as they meet the legislatively defined purposes and parameters. 


24See Pub. L. No. 93-638, 88 Stat. 2203 (1975) (codified as amended at 25 U.S.C. §§ 
5301-5423). 
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Performance 
Measurement and 
Evaluation of Federal 
Grant Programs 


propose a change to the self-determination contract or annual funding 
agreement in order to redesign programs included in the contract.25 


Agencies may assess the performance of their grant programs through 
activities such as performance measurement and evaluation. 
Performance measurement is the ongoing monitoring and reporting of 
program accomplishments, particularly progress toward preestablished 
goals. Program evaluations are individual systemic studies conducted 
periodically or on an ad hoc basis to assess how well a program is 
working. Both forms of assessment aim to support resource allocation 
and other policy decisions to improve service delivery and program 
effectiveness. Because of its ongoing nature, performance measurement 
can serve as an early warning system to agencies if a program is at risk 
of not achieving its objectives. By contrast, a program evaluation typically 
involves a more in-depth examination of program performance than 
performance measurement allows for, and can also identify adjustments 
that may improve programmatic results. 26 


We have previously reported as well as issued guidance on aspects of 
performance measurement and program evaluation.2” Additionally, the 
Office of Management and Budget’s Uniform Administrative 
Requirements, Cost Principles, and Audit Requirements for Federal 
Awards contains requirements for performance monitoring and reporting, 
while external entities such as the American Evaluation Association have 
issued guidance to federal agencies on integrating evaluations into 
program management.28 Furthermore, the Foundations for Evidence- 
Based Policymaking Act of 2018 (Evidence Act), enacted in January 


25In January 2019, we reported on differences between the self-determination contracts 
and self-governance compacts that Interior enters into with tribes and tribal organizations. 
See GAO, /ndian Programs: Interior Should Address Factors Hindering Tribal 
Administration of Federal Programs, GAO-19-87 (Washington, D.C.: Jan. 3, 2019). 


26For the purposes of this review, we use the definitions for performance measurement 
and program evaluation contained within GAO-11-646SP. 


27For selected examples, see GAO, Managing for Results: Further Progress Made in 
Implementing the GPRA Modernization Act, but Additional Actions Needed to Address 
Pressing Governance Challenges, GAO-17-775 (Washington, D.C.: Sept. 29, 2017); and 
Designing Evaluations: 2012 Revision, GAO-12-208G (Washington, D.C.: Jan. 31, 2012). 


28For selected examples, see Office of Management and Budget, Uniform Administrative 
Requirements, Cost Principles, and Audit Requirements for Federal Awards, 2 C.F.R. part 
200; and American Evaluation Association, An Evaluation Roadmap for a More Effective 
Government (Washington, D.C.: Revised October 2016). 2 C.F.R. part 200 generally 
applies to federal grant programs. 
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2019, seeks to create a framework for a more comprehensive and 


integrated approach to federal evidence-building activities.29 Evidence is 
broadly defined and can include activities such as performance 
measurement and program evaluation, according to Office of 
Management and Budget guidance to federal agencies responsible for 
implementing the Evidence Act.°° In March 2020, the Office of 
Management and Budget, in accordance with the Evidence Act, issued 
guidance for program evaluation and identified best practices for 
evaluation that it strongly encourages agencies to consider 
implementing. 3" 


Agencies Provided at 
Least 38 Grant 
Programs Targeted to 
Tribes That Could 
Help Prevent or 
Address Delinquency 
among Native 
American Youth 


29See Pub. L. No. 115-435, 132 Stat. 5529 (enacted Jan. 14, 2019). Some of the specific 
provisions of the Evidence Act include a requirement that agencies appoint a senior 
employee as an evaluation officer. This official would be responsible for activities such as 
assessing an agency’s ongoing evaluation activities and capacity to support the 
development and use of evaluation as well as establishing and implementing an agency 
evaluation policy. We did not examine agencies’ implementation of those activities 
because the scope of our review focused on agencies’ performance measurement or 
program evaluation activities from fiscal years 2015 through 2018. 


30See Office of Management and Budget, Phase 1 Implementation of the Foundations for 
Evidence-Based Policymaking Act of 2018: Learning Agendas, Personnel, and Planning 
Guidance, OMB Memorandum M-19-23 (Washington, D.C.: 2019). 


31See Office of Management and Budget, Phase 4 Implementation of the Foundations for 


Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices, 
OMB Memorandum M-20-12 (Washington, D.C.: 2020). 
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HHS, DOJ, Education, and 
Interior Administered at 
Least 38 Relevant 
Discretionary and Formula 
Grant Programs 


We found that HHS, DOJ, Education, and Interior administered at least 38 
grant programs that could have been used to prevent or address 
delinquency among Native American youth. These include at least 33 
discretionary grant programs from fiscal years 2015 through 2018 and 
five formula grant programs for which tribes could apply directly. See 
appendix III for the full list of grant programs we identified. HHS, DOJ, 
Education, and Interior awarded approximately $1.9 billion through these 
grant programs during this period,?2 with HHS accounting for 
approximately two-thirds of the total amount awarded.33 Figure 2 below 
shows the total number and value of grant programs we identified by 
agency. 4 


32This total does not include funding tribes and tribal organizations received through 
Interior and HHS Indian Health Service self-determination contracts and self-governance 
compacts. 


33For the four HHS formula grants we identified as relevant for this review, we included 
only the funding that the agency awarded directly to tribes and tribal organizations in the 
$1.9 billion figure we cited. 


34This amount includes both initial funding agencies award to grantees upon approving 
their application as well as any continuation awards that agencies award to grantees in 
subsequent years. A continuation award is an extension or renewal of existing program 
funding for one or more additional period(s) that would otherwise expire and are typically 
available to existing grantees of discretionary, multiyear projects. The fiscal year 2018 
funding opportunity announcement for the HHS Tribal Behavioral Health Grant Program, 
for example, informs applicants that they could receive up to $250,000 per year for up to 5 
years, depending on conditions such as available funds, progress in meeting project 
goals, and timely reporting. 
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HHS 


Figure 2: Number and Value of Selected Grant Programs That Grantees Could Have 
Used to Help Prevent or Address Tribal Juvenile Delinquency, Fiscal Years 2015- 
2018 


Agency Total value of selected 
grant programs 


Department of Education $548.9 million 


Department of Health and 


Human Services $1.3 billion 
Department of the Interior $4 million 
Department of Justice $74.6 million 
0 5 10 15 20 25 


Number of grant programs 
Source: GAO analysis of federal grant award data. | GAO-20-600 
Note: These totals do not include funding tribes and tribal organizations received through Department 


of the Interior and Department of Health and Human Services’ (HHS) Indian Health Service self- 
determination contracts and self-governance compacts. 


*For the HHS formula grants we identified as relevant for this review, we included only the funding 
that the agency awarded directly to tribes and tribal organizations. 


HHS administered 21 of the 33 discretionary grant programs and four of 
the five formula grant programs that we identified, totaling 25 grant 
programs valued at approximately $1.3 billion.35 For example, HHS’s 
Indian Health Service solicited applications for the Methamphetamine and 
Suicide Prevention Initiative-General Indigenous Initiative Support 
program in fiscal years 2016 and 2017. This discretionary grant program 
seeks culturally appropriate ways to prevent methamphetamine and other 
substance use that could contribute to suicidal behaviors; the grant also 
seeks to hire additional behavioral health staff in tribal communities that 
specialize in child and family services. One of the HHS formula grant 
programs we identified is the Promoting Safe and Stable Families 
program, whose primary goals include preventing the unnecessary 
separation of children from their families. This program reserves 3 


35HHS’s Promoting Safe and Stable Families grant program we identified as relevant for 
this review contains award funding on both a formula and discretionary basis. However, 
for the purposes of this review, we characterize it as a formula grant because HHS 
primarily obligates program awards on a formula basis. For example, of the approximately 
$327 million HHS obligated for the Promoting Safe and Stable Families program in fiscal 
year 2018, HHS obligated only about $2 million in discretionary awards. 
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DOJ 


percent of its total annual funding for eligible tribes and tribal consortia. 26 
See figure 3 below for the HHS award amounts by component office. 


Figure 3: Number and Value of Department of Health and Human Services Grant 
Programs That Could Help Prevent or Address Tribal Juvenile Delinquency by 
Component Office, Fiscal Years 2015-2018 


Component Office Total value of selected 
grant programs 
Administration for $1.0 billion? 


Children and Families 


Substance Abuse and 
Mental Health Services $185.9 million 
Administration 


Indian Health Service $82.2 million 


National Institutes 


of Health $5.0 million 


Office of Minority Health $2.3 million 


0 3 6 9 12 15 


Number of grant programs 


Source: GAO analysis of Department of Health and Human Services grant award data. | GAO-20-600 


Note: The these totals do not include funding tribes and tribal organizations received through 
Department of Health and Human Services’ Indian Health Service self-determination contracts and 
self-governance compacts. 

*For the Administration for Children and Families (ACF) formula grants we identified as relevant for 
this review, we included only the funding that the agency awarded directly to tribes and tribal 
organizations. For example, ACF awarded tribal grantees approximately $42 million for the Promoting 
Safe and Stable Families formula grant program from fiscal years 2015 through 2018 out of a total 
amount of approximately $1.5 billion during this period. The $1 billion cited above for ACF only 
includes the $42 million it awarded to tribal grantees for this formula grant program and not the entire 
$1.5 billion it awarded to all of the program grantees. 


DOJ administered eight of the 33 discretionary grant programs we 
identified that could have helped prevent or address delinquency among 


36In fiscal year 2017, for example, total funding for the Promoting Safe and Stable 
Families program was about $381 million, and tribal entities received approximately $11.5 
million, or about 3 percent, of the program’s total funding. 
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tribal youth, totaling approximately $74.6 million.3” For example, OJJDP 
administered the Coordinated Tribal Assistance Solicitation Purpose Area 
9 Tribal Youth Program each fiscal year from 2015 through 2018. This 
program provides funding for a range of prevention, intervention, and 
treatment activities that support the overall goals of addressing juvenile 
delinquency in tribal communities and strengthening the juvenile justice 
system with which Native American youth come into contact.38 See figure 
4 for the DOJ award amounts by grant program. 


EEE ees 
Figure 4: Value of Selected Department of Justice (DOJ) Grant Programs That 


Could Help Prevent or Address Tribal Juvenile Delinquency, Fiscal Years 2015-2018 
Grant program 
CTAS Purpose Area 4: 


Tribal Justice Systems 
Infrastructure Program 


CTAS Purpose Area 9: 
Tribal Youth Program 


CTAS Purpose Area 8: 
Tribal Juvenile Healing 
to Wellness Courts 


Tribal Youth Program Training 
and Technical Assistance 


Tribal Defending Childhood 
Demonstration Program 


Supporting Tribal Youth: 
Training, Technical Assistance, 
and Youth Leadership 


Alaska Native Youth Training and 
Technical Assistance Project 


0 5 10 15 20 25 30 35 
Total value (dollars in millions) 


Legend 
CTAS= Coordinated Tribal Assistance Solicitation 


Source: GAO analysis of DOJ grant award data. | GAO-20-600 


Note: We identified the National Intertribal Youth Leadership Development Initiative, for which the 
Office of Juvenile Justice and Delinquency Prevention issued a solicitation inviting applicants to apply 


37We identified the National Intertribal Youth Leadership Development Initiative as within 
our scope. However, DOJ officials told us that the agency did not make any awards for the 
program. Also, DOJ’s Title Il Formula Grants Program provides funding to states to 
develop programs to address delinquency and improve the juvenile justice system. 
However, we did not select this program for our review because tribal entities are not 
eligible to apply directly for funding. 


38The Coordinated Tribal Assistance Solicitation is a mechanism by which DOJ, since 
fiscal year 2010, has combined existing tribal—specific competitive solicitations into one 
solicitation and required only one application from each tribe or tribal consortium. 
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for funding in fiscal year 2017, as a grant program that could have helped prevent or address 


Education 


delinquency among Native American youth. However, DOJ officials told us that the agency received 
three applications but did not make any awards for the program. 


We identified three discretionary and one formula grant program that 
Education administered from fiscal years 2015 through 2018 totaling 
about $548.9 million.39 For example, Education’s Office of Elementary 
and Secondary Education offered funding for up to 3 years for the Native 
American Language Program in fiscal years 2017 and 2018. This 
program supports schools that use Native languages as the primary 
language of instruction and supports the rights and ability for tribal 
communities to use, practice, maintain, and revitalize their languages, 
among other purposes. According an OJJDP literature review, culture— 
which includes traditional values, customs, activities, ceremonies, and 
language—can be a protective factor that could potentially increase 
resiliency among Native American youth and reduce the likelihood of 
negative outcomes such as contact with the juvenile justice system.4° The 
Office of Elementary and Secondary Education administers the Indian 
Education Formula Grants to Local Educational Agencies formula grant 
program. This program supports the efforts of entities such as local 
educational agencies and tribes in developing elementary school and 
secondary school programs for Native American students that are 
designed to meet their unique cultural, linguistic, and educational needs. 
According to Education officials, the Office of Elementary and Secondary 
Education made over 1,200 awards to grantees through the Indian 
Education Formula Grants to Local Education Agencies program each 
fiscal year from 2015 through 2018. See figure 5 below for the Education 
program-by-program award amounts. 


39Education officials told us that although the agency grant programs cited in the report 
could reduce student interactions with the justice system, doing so is not the statutory 
purpose of these programs. 


40See Office of Juvenile Justice and Delinquency Prevention, Literature Review: A 


Product of the Model Programs Guide—Tribal Youth in the Juvenile Justice System 
(Washington, D.C.: Development Services Group, Inc., 2016). 
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Interior 


Interior and HHS 
Administered Self- 
Determination Contracts 
and Self-Governance 
Compacts That Provide 
Financial Assistance to 
Tribes 


Figure 5: Value of Selected Department of Education Grant Programs That Could 
Help Prevent or Address Tribal Juvenile Delinquency, Fiscal Years 2015-2018 


Grant program 


Indian Education Formula Grants 
to Local Educational Agencies 


Indian Education Demonstration Grants— 
Native Youth Community Projects 


Native American and Alaska Native Children 
in School Program 


Native American Language Program 
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Source: GAO analysis of Department of Education grant award data. | GAO-20-600 


We identified one Interior grant program the agency administered from 
fiscal years 2015 through 2018 totaling about $4 million that could help 
prevent or address delinquency among Native American youth—the 
Native Language Immersion Cooperative Agreement. Interior's BIE made 
awards in fiscal years 2017 and 2018 to grantees through this 
discretionary grant program to expand existing language immersion 
programs or create new programs that will lead to oral proficiency in tribal 
languages within BIE-funded schools. Interior officials told us that in fiscal 
year 2017, BIE awarded about $2 million to five universities and other 
organizations focused on Native language preservation or education to 
assist it in creating tribal language-specific professional development, 
classroom, and curriculum resources for teachers in BIE-funded schools. 
According to these officials, BIE again made $2 million in awards through 
the Native Language Immersion Cooperative Agreement in fiscal year 
2018, this time to 15 BIE-funded schools. 


Interior and HHS’s Indian Health Service also provided financial 
assistance to tribes through self-determination contracts and self- 
governance compacts from fiscal years 2015 through 2018 that tribes 
used to help prevent or address juvenile delinquency in their 
communities, according to Indian Health Service and Interior officials. 
Because the funding that tribes receive from agencies for self- 
determination contracts and self-governance compact is not a grant, the 
monitoring and performance standards that generally apply to federal 
grant programs do not generally apply to these agreements. 
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HHS officials told us that the Indian Health Service, for example, provides 
regular funding to 12 Youth Regional Treatment Centers, half of which the 
Indian Health Service operates directly while tribes or tribal organizations 
operate the other half under funding from self-determination contracts or 
self-governance compacts.‘*1 These treatment centers seek to provide 
comprehensive services for Native American youth and their families in a 
residential environment that integrates traditional healing, spiritual values, 
and cultural identification. Some of the services in the Youth Residential 
Treatment Centers include substance abuse education and art therapy, 
as well as aftercare relapse prevention and post-treatment follow-up 
activities. In addition to these Youth Regional Treatment Centers, we 
found, for example, that a tribe in South Dakota chose to use funding 
from its self-determination contract with the Indian Health Service in fiscal 
year 2017 to implement rehabilitation and aftercare services for youth 
within its community that were struggling with addiction. Some of the 
rehabilitation services that this tribe provided included assessments, 
referrals, and equine (horse)-assisted psychotherapy, while some of the 
aftercare services it provided for youth in its community included 
improving decision-making and developing coping skills. 


Officials from Interior described various activities related to preventing or 
addressing juvenile delinquency that tribes implemented using funding 
they received through self-determination contracts and self-governance 
compacts.‘ These activities included recidivism programs, tribal courts, 
and juvenile detention services. For example, Interior officials stated that 
some tribes included a program called the Recidivism Reduction Initiative 
in their self-determination contracts or self-governance compacts. The 
purpose of the Recidivism Reduction Initiative is to identify individuals 
incarcerated at adult and juvenile detention facilities in Indian country who 


“According to HHS officials, the Indian Health Service entered into an average of 248 
self-determination contracts between fiscal years 2015 and 2018 worth an average total 
value of $803 million per fiscal year during this period. For self-governance compacts, 
Indian Health Service officials told us that their office entered into an average of 119 
funding agreement per fiscal year worth an average total value of $2 billion per fiscal year 
during this period. As we mentioned earlier, self-determination contracts and self- 
governance compacts allow tribes or tribal organizations to assume administration of 
certain federal programs—or portions thereof—that the HHS’s Indian Health Service or 
Interior previously managed. 


42According to Interior officials, the agency entered into an average of 731 self- 
determination contracts between fiscal years 2015 and 2018 worth an average total value 
of $553 million per fiscal year during this period. For Interior’s self-governance compacts, 
officials told us that the agency entered into an average of 116 funding agreements per 
fiscal year worth an average total value of $487 million per fiscal year during this period. 
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When Assessing 
Selected Grant 
Programs, Agencies 
Incorporated Almost 
All Leading Practices 
for Performance 
Measurement and 
Program Evaluation 
All Agencies in Our 
Review Assessed the 
Performance of Grant 
Programs through 
Performance 
Measurement, and HHS, 


DOJ, and Education Used 
Program Evaluations 


Performance measurement 


are at high risk for relapse into criminal or delinquent behavior and then to 
follow-up with them with post-intervention services for 1 year. Interior 
officials told us that five tribes received $1.5 million in Recidivism 
Reduction Initiative funding in fiscal year 2018 and that two of the five 
tribes decided to focus their activities on juvenile recidivism. Additionally, 
some tribes have included funding from the Johnson-O’Malley Program in 
their self-determination contracts or self-governance compacts for 
education services that could help prevent or address juvenile 
delinquency, such as Native language classes and dropout prevention 
programs.‘ 


HHS, DOJ, Education, and Interior assessed the performance of grant 
programs through performance measurement, while HHS, DOJ, and 
Education assessed the performance of some of their grant programs 
through program evaluations. In assessing the performance of selected 
grant programs and the extent to which component offices incorporated 
leading practices, we found these four agencies incorporated almost all 
leading practices we identified for performance measurement and 
program evaluation. 


HHS, DOJ, Education, and Interior assessed the performance of selected 
grant programs they administered from fiscal years 2015 through 2018 in 
part by conducting a range of performance measurement activities. We 


43See generally 25 C.F.R. part 273. We reported on this program in April 2020. See GAO, 
Bureau of Indian Education: Actions Needed to Improve Management of a Supplemental 
Education Program, GAO-20-308 (Washington, D.C.: Apr. 9, 2020). 
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Program evaluation 


found, for example, that all of these agencies monitor and receive 
updates on the extent to which their grantees are meeting the goals of a 
given grant program by requiring them to submit periodic performance 
reports—either quarterly, semiannually, or annually.44 Agency officials for 
grant programs such as HHS’s Native American Language Preservation 
and Maintenance-Esther Martinez Immersion then explicitly certify, upon 
reviewing performance reports, that grantees are making progress on key 
activities and objectives. Officials from some agency component offices, 
such as Education’s Office of English Language Acquisition, told us that 
they use annual performance reports to identify technical assistance 
needs for grantees, particularly in instances where grantees struggle to 
collect and submit performance data. 


In addition, selected component offices from these agencies conduct 
other activities to assess the performance of selected grant programs 
such as site visits, emails, conference calls, or desk reviews of grantees’ 
project documentation. For example, DOU’s Office of Justice Programs 
conducts desk reviews where grant managers remotely analyze the 
financial, administrative, and programmatic soundness of a grant program 
and identify any noncompliance in areas of concern to address with the 
grantee, among other actions. These grant managers then assess and 
certify that grantees are current with reporting performance measurement 
data and that grantees’ activities or deliverables support project goals and 
objectives, among other activities. 


HHS, DOJ, and Education conducted or sponsored relevant program 
evaluations to assess the performance of some of their grant programs 
between fiscal years 2015 and 2018. Generally, these agencies entered 
into contracts with or made awards to organizations to independently 
conduct program evaluations on their behalf. 


HHS. Within HHS, the Office of the Assistant Secretary for Planning and 
Evaluation, as well as HHS component offices, conduct or sponsor 
program evaluations.45 We identified one program evaluation HHS 
published from fiscal years 2015 through 2018 for a grant program we 


44We also found that these agencies required grantees to submit financial reports. 


45The HHS component offices for which we identified grant programs relevant for this 
review include the Administration for Children and Families (ACF), Indian Health Service, 
National Institutes of Health, the Office of Minority Health, and the Substance Abuse and 
Mental Health Services Administration. 
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selected as relevant for this review.46 Specifically, the HHS Administration 
for Children and Families’ (ACF) Office of Planning, Research and 
Evaluation awarded a contract to an organization to conduct a cross-site 
evaluation of the Project Linking Actions for Unmet Needs in Children’s 
Health program, which it published in April 2015.47 This grant program, 
which the Substance Abuse and Mental Health Services Administration 
administers, seeks to promote the wellness of young children by 
addressing the social, emotional, cognitive, physical, and behavioral 
aspects of their development. In addition, in April 2020, HHS officials 
stated that the agency is in the process of developing an approach to 
implement the Evidence Act, which aims in part to improve agencies’ 
federal evidence-building activities, including program evaluation 
capacity. HHS officials told us the agency submitted its planned approach 
for implementing the Evidence Act to the Office of Management and 
Budget in September 2019. This approach includes developing a process 
to collect information from component offices to create office-specific 
evidence-building plans and annual evaluation plans. 


DOJ. We identified five program evaluation reports that DOJ published 
from fiscal years 2015 through 2018 that focused on Native American 
youth. The National Institute of Justice, DOJ’s research, development, 
and evaluation office, awarded funding for four of these, while OJJDP, in 
coordination with the Library of Congress, awarded funding for the fifth. 
See table 1 below for the list of these reports. 


46|n 2016, HHS’s ACF awarded a contract to two organizations to conduct a multisite 
program evaluation on Tribal Home Visiting programs, which account for two grant 
programs we selected as relevant for our review. ACF plans to complete this program 
evaluation by 2021. In addition, HHS officials told us in April 2020 that the Indian Health 
Service will use the aggregate data that it had previously collected for the 
Methamphetamine and Suicide Prevention Initiative - Generation Indigenous Initiative 
Support program to complete evaluation of the program in 2021. 


47See Abt Associates, Implementation of Project LAUNCH: Cross-Site Evaluation 
Findings, Volume |, a report prepared at the request of the Department of Health and 
Human Services, Administration for Children and Families, Office of Planning, Research 
and Evaluation (Rockville, MD: December 2014); and Outcomes of Project LAUNCH: 
Cross-Site Evaluation Findings, Volume II, a report prepared at the request of the 
Department of Health and Human Services, Administration for Children and Families, 
Office of Planning, Research and Evaluation (Rockville, MD: December 2014). 
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Table 1: Department of Justice (DOJ)-Awarded Program Evaluation Reports That Focused on Native American Youth 
Published between Fiscal Years 2015 and 2018 


Program evaluation report title 


DOJ component office that awarded Publication 
funding for program evaluation report date 


Cross-Site Evaluation of the Office of Juvenile Justice and Delinquency Office of Juvenile Justice and Delinquency December 2014 


Prevention Tribal Green Reentry Program: Final Technical Report Prevention 

Protect, Heal, Thrive: Lessons Learned from the Defending Childhood National Institute of Justice May 2015 
Demonstration Program 

Love One Another and Take Care of Each Other: A Process Evaluation National Institute of Justice May 2015 
of the Rocky Boy’s Children Exposed to Violence Project 

Nawicakiciji-Woasniye-Oaye Waste: A Process Evaluation of the National Institute of Justice June 2015 
Rosebud Sioux Tribe’s Defending Childhood Initiative 

An Outcome Evaluation of the Defending Childhood Demonstration National Institute of Justice November 2015 
Program 


Source: DOJ. | GAO-20-600 


As of March 2020, DOJ officials told us that the agency is in the process 
of implementing Office of Management and Budget guidance related to 
the Evidence Act but has yet to determine how evaluation activities that 
focus on Native American youth will factor into DOJ’s annual evaluation 
plan under the act.48 DOJ has also designated an evaluation officer to 
coordinate the agency’s evidence-building activities, as required by the 
Evidence Act. 


Education. We found that Education published one program evaluation 
from fiscal years 2015 through 2018 for one of the grant programs we 
identified as relevant for our review. Specifically, Education’s Office of 
Planning, Evaluation, and Policy Development awarded a contract to an 
organization to conduct a program evaluation on the Native American and 
Alaska Native Children and School Program, which it published in June 


48The Office of Management and Budget issued guidance to federal agencies on 
implementing the Evidence Act in July 2019 that recommended agencies develop annual 
evaluation plans as a component of an agency’s learning agenda. According to this 
guidance, the creation of learning agendas requires agencies to identify and set priorities 
for evidence-building, in consultation with various stakeholders. See OMB Memoranda M- 
19-23 and M-20-12. 
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2018.49 In March 2020, Education officials told us that the agency intends 
to publish special analyses of services provided for Native American 
students and by tribal organizations when it is possible to do so, as part of 
evaluations of Titles | through IV of the Elementary and Secondary 
Education Act, as amended by the Every Student Succeeds Act. 
Education officials also stated that the agency is developing a learning 
agenda under the Evidence Act, which will seek to develop an 
understanding of policies and practices that can improve the education 
and outcomes of different groups of students.5° As of March 2020, 
officials also stated that Education is developing an agency-wide 
evaluation policy, in accordance with the Evidence Act. 


Interior. Interior's BIE did not conduct or publish any relevant program 
evaluations from fiscal years 2015 through 2018, citing the bureau’s 
staffing shortages and limited capacity as reasons why it did not do so. 
However, in March 2020, a BIE official told us that the bureau had 
recently hired an official to manage the office that administers the Native 
Language Immersion Cooperative Agreement. This management official 
told us he has started to meet with contractors to understand the activities 
they are delivering for the Native Language Immersion Cooperative 
Agreement. This official also was able to describe the activities the office 
would need to implement to position itself to conduct evaluations of the 
grant program, including hiring additional staff; training the staff in 
evaluation; holding discussions with subject matter experts; and 
developing an evaluation framework, instrument, and procedures. 


49See American Institutes for Research, Study of the Native American and Alaska Native 
Children in School Program: FY 2011 and FY 2013 Cohorts, a report prepared at the 
request of the Department of Education, Office of Planning, Evaluation, and Policy 
Development, Policy and Program Studies Service (Washington, D.C.: June 2018). 
According to Education officials, independent evaluations of agency programs are also 
conducted through contracts to research firms and overseen by its Institute of Education 
Sciences. Education’s Office of Planning, Evaluation, and Policy Development more 
recently published the program evaluation focused on Native American youth, entitled 
“Study of the Implementation of the Title VI Indian Education Formula Grants Program” in 
October 2019. 


50Education officials told us that following the passage of the Evidence Act, an Evidence 
Leadership Group has centrally coordinated the agency’s evidence-building agenda. 
According to officials, this group includes representatives from all of the agency’s principal 
offices and is co-chaired by the agency's evaluation officer and Director of the grants 
policy office. 
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HHS, DOJ, Education, and 
Interior Incorporated 
Almost All Leading 
Practices We Identified for 
Performance 
Measurement or Program 
Evaluation 


HHS Component Offices 
Incorporated Almost All 
Leading Practices We 
Identified for Performance 
Measurement 


Four of the five HHS component offices that administered selected grant 
programs incorporated all of the leading practices we identified for 
performance measurement.*' A fifth HHS component office, ACF, fully 
incorporated all but one of the leading practices for performance 
measurement.*2 Specifically, the Indian Health Service, National Institutes 
of Health, Office of Minority Health, and the Substance Abuse and Mental 
Health Services Administration incorporated all 14 leading practices for 
performance measurement. For example, all four of these HHS 
component offices incorporated the leading practice of results being 
reported periodically throughout a program’s life cycle by requiring 
grantees to submit semiannual or annual performance reports as they are 
implementing their projects as well as final performance reports at the 
end of their projects. However, we found that although ACF incorporated 
13 of the 14 leading practices for performance measurement we 
identified, the component office did not fully incorporate the leading 
practice related to assessing grantee performance data for quality and 
reliability for the Stephanie Tubbs Jones Child Welfare Services Program 
(see table 2). 


51The HHS component offices administered eight of our 15 selected grant programs. 


52We did not assess the extent to which HHS incorporated leading practices for program 
evaluation because we did not identify an evaluation report that HHS’s Office of the 
Assistant Secretary for Planning and Evaluation or relevant HHS component offices 
published from fiscal years 2015 through 2018 that focused on Native American youth. 
Although ACF published a two-volume program evaluation report on the Project Linking 
Actions for Unmet Needs in Children's Health program in April 2015, we did not assess 
the extent to which the HHS component office incorporated leading practices we identified 
for program evaluation because this report concerned grantees that we determined likely 
fell outside the scope of our review. Specifically, only seven of the 40 grant recipients 
were tribal entities, and only one of the 24 grantees on which the findings of these reports 
were based was a tribal entity. We generally could not determine which findings applied to 
that tribal entity because these evaluations generally reported findings in the aggregate. 
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Table 2: Assessment of Department of Health and Human Services Administration for Children and Families Incorporation of 
GAO-Identified Leading Practices for Performance Measurement 


Administration for 
Leading practices for performance measurement Children and Families 


1. The agency should establish activities to measure performance that compare program achievements 


to program objectives. . 
The agency should have performance plans that outline required performance measurement activities. @ 
The agency should incorporate risk assessment in its performance measurement activities. e 
4. Performance measurement activities should be aligned with the needs of users of performance e 
information. 
5. Management should use performance data when making programmatic and operational decisions. @ 
6. Results should be clearly communicated to users and stakeholders. e 
7. Results should be reported periodically throughout a program’s life cycle. e 
8. Performance goals and measures should cover core program activities and priorities. e 
9. Performance goals and measures should align with agency and/or government-wide priorities. e 
10. Performance goals and indicators should be measurable, objective, and reliable. e 
11. Performance goals and measures should clearly communicate performance targets. @ 
12. Performance measures should provide baseline data and measure program progress. @ 
13. Data should be assessed for quality and reliability. © 
14. The agency should clearly communicate the scope and timing of performance measurement activities e 


to grantees. 


Legend: 
@ = The performance measurement activity incorporated the leading practice. 
© = The performance measurement activity partially incorporated the leading practice. 


O = The performance measurement activity did not incorporate the leading practice. 
Source: GAO. | GAO-20-600 


Officials from the ACF Children’s Bureau told us that they do not attempt 
to independently validate the reliability of tribal grantee performance data 
for the Stephanie Tubbs Jones Child Welfare Services Program. 
Children’s Bureau officials told us that they have regular communications 
with tribal grantees and provide technical assistance to them, in addition 
to reviewing data that grantees of the Stephanie Tubbs Jones Child 
Welfare Services Program submit in their annual performance reports. 
However, these officials stated that the Children’s Bureau does not 
attempt to independently validate the reliability of tribal grantee 
performance data for this program. These officials explained that if, for 
example, a tribal grantee submitted notably different performance data 
from year to year, they would follow up to obtain more information. 
However, they said they are not required to do so. Officials also said that 
they do not assess or validate grantee data for the Stephanie Tubbs 
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DOJ Component Offices 
Incorporated All Leading 
Practices for Performance 
Measurement and Almost All 
for Program Evaluation 


Jones Child Welfare Services Program because, since the program is a 
formula grant program, funding decisions for grantees are not tied to 
performance metrics. 53 


Leading practices for performance measurement we identified include 
agencies assessing grantee performance data for quality and reliability. 
Assessing data reliability could include activities such as asking selected 
questions to grantees or requesting documentation from them about, for 
example, how they collect data, or grantees’ opinion of the quality of the 
data they are providing, and then reviewing these responses and 
documents.54 Developing a process to assess the reliability of tribal 
grantee performance data for the Stephanie Tubbs Jones Child Welfare 
Services Program, specifically in the annual performance reports that 
tribal recipients submit, could help provide the ACF Children’s Bureau 
with further assurance that it has an accurate representation of grantee 
performance. Furthermore, while grantee performance data will not affect 
funding decisions for a formula grant program, having such reliable 
information could enable the ACF Children’s Bureau to identify and 
address any potential grantee programmatic challenges that might 
emerge. 


We found the DOJ component offices in our review incorporated all 14 
leading practices for performance measurement we identified and 13 of 
14 leading practices for program evaluation. For example, DOJ’s Office of 
Justice Programs, which administered all of the agency’s grant programs 
we selected, incorporated leading practices such as clearly 
communicating the scope and timing of performance measurement 
activities to grantees by providing such information in both funding 
opportunity announcements that invite applicants to apply for federal 


53As previously mentioned, formula grants are a type of noncompetitive grant that 
agencies award to grantees based on statistical criteria. Authorizing legislation and 
regulations usually define the statistical criteria and the funding amount allocated to 
recipients of formula grants. 


54Other examples of potential data reliability activities could include testing data for 
missing, duplicate, or unexpected values or tracing a sample of data records to or from 
source documents to identify any gaps. See GAO, Assessing Data Reliability, 
GAO-20-283G (Washington, D.C.: Dec. 16, 2019). 
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funding as well as grant award notices.55 The National Institute of Justice, 
which funded four of the five program evaluation reports we reviewed, 
incorporated 13 of the 14 leading practices we identified for program 
evaluation.5¢ For example, the National Institute of Justice incorporated 
leading practices such as disseminating evaluation findings to 
stakeholders and the public when possible by posting program evaluation 
reports onto its publications webpage, as well as submitting the reports 
for posting to the National Archive of Criminal Justice Data website.57” 
However, the agency did not fully incorporate the leading practice for 
program evaluation we identified related to ensuring that program 
evaluation reports include a discussion of their limitations. See table 3 
below for more details. 


55DOJ’s Office of Justice Programs administered three of our 15 selected grant programs. 
According to Grants.gov, a notice of award is the official and legally binding issuance of 
award funding that obligates grantees to carry out the terms and conditions of the grant 
program. 


56A though OJJDP awarded funding for one of the program evaluation reports that we 
identified as relevant for this review, we did not assess its program evaluation practices at 
the component office level because DOJ officials informed us that OJJDP’s research unit 
that funded program evaluation reports had disbanded and transferred its functions to the 
National Institute of Justice in the summer 2018. 


57The mission of the National Archive of Criminal Justice Data, located at the University of 
Michigan, is to archive and disseminate data on criminal justice issues. Various DOJ 
offices, such as the Bureau of Justice Statistics, the National Institute of Justice, and 
OJJDP sponsor the National Archive of Criminal Justice Data. 
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Table 3: Assessment of the Department of Justice National Institute of Justice’s Incorporation of GAO-Identified Leading 
Practices for Program Evaluation 


Leading practices for program evaluation 


GAO Assessment 


1. The agency should have evaluation plans. e 

2. The agency should have written policies to guide evaluation activities and e 
establish quality standards. 

3. Evaluation policies should require evaluators to have the necessary knowledge e 
and competencies for the evaluation. 

4. Evaluation policies should have ethical behavior guidelines, including ensuring e 
evaluator independence. 

5. Evaluation findings should be disseminated to stakeholders and the public, e 
when possible. 

6. Management should use evaluation findings to make programmatic and e 
operational decisions. 

7. Management should use evaluations to identify innovative or promising e 
practices. 
Evaluators should involve a full range of stakeholders in the planning process. e 

9. Evaluation should use the most rigorous methods appropriate to the questions e 
and context. 

10. Evaluation questions should be feasible, relevant, clearly defined, and e 
measureable. 

11. Data should be assessed for quality and reliability. e 

12. The evaluation’s conclusions should be supported by the findings. e 

13. The criteria and measures should be observable and appropriate to the e 
evaluation questions. 

14. The evaluation report should include a discussion of the study’s limitations. © 

Legend: 


@ = The program evaluation activity incorporated the leading practice. 


© = The program evaluation activity partially incorporated the leading practice. 


O = The program evaluation activity did not incorporate the leading practice. 
Source: GAO. | GAO-20-600 


Two National Institute of Justice-funded program evaluation reports, both 
published in 2015, did not include a discussion of possible limitations 
related to their questions, methods, findings, or conclusions. However, 
since 2017, the National Institute of Justice has included in its final 
research report guidelines that reports should include a discussion of 
limitations. In March 2020, DOJ officials confirmed that the National 
Institute of Justice attaches a special condition to its research, evaluation, 
and development-related awards directing grantees to use the final 
research report guidelines on the National Institute of Justice website. 
The special condition states that the National Institute of Justice expects 
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Education Component Offices 
Incorporated All Leading 
Practices We Identified for 
Performance Measurement 
and Program Evaluation 


Interior's BIE Incorporated 
Most but Not All Leading 
Practices We Identified for 
Performance Measurement 


grantees to contact their grant manager if they have any question or need 
clarification regarding the form and content of their final research report. 


Education component offices incorporated all the leading practices we 
identified for performance measurement and program evaluation. 
Specifically, the Office of Elementary and Secondary Education and the 
Office of English Language Acquisition, which administered three of our 
selected 15 grant programs, incorporated all 14 leading practices for 
performance measurement. For example, both offices incorporated the 
leading practice of employing risk assessment into their performance 
measurement activities by using an agency-wide tool that provides 
administrative, financial, and internal control information on discretionary 
grantees to determine if they have a low, elevated, or significant potential 
risk in these domains. The agency then proposed various mitigation 
strategies for addressing the potential administrative, financial, or internal 
control risk that a grantee might pose. Additionally, Education’s Office of 
Planning, Evaluation, and Policy Development, which published the 
agency’s program evaluation on the Native American and Alaska Native 
Children in School Program, incorporated the 14 leading practices we 
identified for program evaluation. For example, the Office of Planning, 
Evaluation, and Policy incorporated the leading practice of having policies 
to guide evaluation activities and establish quality standards by 
developing guidance in November 2017 that lists and describes in detail 
the five quality standards that all office-funded program evaluations must 
meet.58 


BIE fully incorporated 11 of the 14 leading practices we identified for 
performance measurement.5? For example, BIE incorporated the leading 
practice we identified that a program’s performance goals and measures 
should align with agency- or government-wide priorities for the one 
relevant Interior grant program in our review—the Native Language 
Immersion Cooperative Agreement. Specifically, officials told us that BIE 
issued the funding opportunity announcement to invite applicants to apply 
for grant program funding as part of a broader government effort to 
supplement Native language programs that Education and HHS already 


58These five quality standards include (1) relevant, (2) accurate, (3) objective, (4) 
comprehensive, and (5) clear. 


59We did not assess the extent to which BIE incorporated leading practices for program 


evaluation because we did not identify an evaluation report it published from fiscal years 
2015 through 2018 that focused on Native American youth. 
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implement.6° However, we also found that BIE did not fully incorporate 
three leading practices we identified for performance measurement 
related to using performance data when making programmatic and 
operational decisions, reporting results throughout a program’s life cycle, 
and assessing performance data for quality and reliability (see table 4 
below for more details). 


cl a gh a a a a 
Table 4: Assessment of the Department of the Interior Bureau of Indian Education’s Incorporation of GAO-Identified Leading 
Practices for Performance Measurement 


Bureau of Indian 
Leading practices for performance measurement Education 


1. The agency should establish activities to measure performance that compare program achievements 
to program objectives. 


2. The agency should have performance plans that outline required performance measurement activities. 


3. The agency should incorporate risk assessment in its performance measurement activities. 


4. Performance measurement activities should be aligned with the needs of users of performance 
information. 


Management should use performance data when making programmatic and operational decisions. 


Results should be reported periodically throughout a program’s life cycle. 


5 
6. Results should be clearly communicated to users and stakeholders. 
7 
8 


Performance goals and measures should cover core program activities and priorities. 


9. Performance goals and measures should align with agency- and/or government-wide priorities. 


10. Performance goals and indicators should be measurable, objective, and reliable. 


11. Performance goals and measures should clearly communicate performance targets. 
12. Performance measures should provide baseline data and measure program progress. 


13. Data should be assessed for quality and reliability. 


14. The agency should clearly communicate the scope and timing of performance measurement activities 
to grantees. 


e\;ee@eeee ef ef © |e @ 


Legend: 
@ = The program evaluation activity incorporated the leading practice. 
© = The program evaluation activity partially incorporated the leading practice. 


O = The program evaluation activity did not incorporate the leading practice. 
Source: GAO. | GAO-20-600 


Using performance data when making programmatic and 
operational decisions. Although BIE uses grantee performance data 
from the Native Language Immersion Cooperative Agreement to make 


60We identified Education’s Native American Language Program and HHS’s Native 
American Language Preservation and Maintenance - Esther Martinez Immersion grant 
programs, both of which focus on Native languages, as relevant for our review. 
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operational decisions, such as planning site visits to grantees, it has not 
used information to make programmatic decisions, such as the possible 
redesign or reallocation of resources within the grant program. In January 
2020, BIE officials explained that they have received a final-year 
performance report from only one grantee thus far and that they plan to 
use grantee performance data to inform programmatic decisions once 
they collect all the grantees’ final-year performance reports. In addition, 
as of April 2020, a BIE official stated that the BIE performance office 
anticipates working with the office that administers the Native Language 
Immersion Cooperative Agreement in identifying performance metrics and 
designing an effective system of data collection, management, and 
analysis to facilitate data-informed decision-making for the program. 


Reporting results throughout a program’s life cycle. BIE did not 
collect performance reports from its Native Language Immersion 
Cooperative Agreement program grantees in a timely manner. 
Specifically, BIE mandated that grantees submit quarterly performance 
reports no later than 30 days after the end of each quarter in their 
project’s period of performance. However, we found that at least two 
fiscal year 2017 grantees of this grant program had not submitted their 
first- or second-quarter performance reports until the period when their 
third-quarter performance reports were due. Additionally, the Interior 
officials managing the grant program did not alert the grantees that they 
were late in submitting their quarterly performance reports until the third 
quarter of their projects’ period of performance. According to a BIE 
program official, staffing shortages made it difficult for the bureau to follow 
up with grantees in a timely manner. Specifically, the official told us that 
BIE had unexpectedly received funding to implement the Native 
Language Immersion Cooperative Agreement and, given the staffing 
shortages, lacked the support staff to manage it that other BIE grant 
programs have. In March 2019, we reported on BIE staffing gaps in our 
2019 High Risk Report, including that about 50 percent of all BIE 
positions had not been filled. However, we also reported that BIE had 
made some progress in increasing bureau capacity by, for example, 
hiring a full-time program analyst to coordinate the bureau’s working 
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group and help oversee the implementation of recommendations 
described in our 2019 High Risk report.® 


One leading practice we identified for performance measurement states 
that agencies should ensure that grantees report performance results 
periodically throughout the life cycle of a grant program. Given BIE’s 
ongoing staffing gaps, taking steps to alert Native Language Immersion 
Cooperative Agreement grantees when they are late in submitting 
performance reports could help the bureau ensure that grantees are 
complying with the terms and conditions of the grant program. Doing so 
could also help BIE officials respond in a timely manner to any 
performance issues during the life cycle of the grant program. 


Assessing performance data for quality and reliability. We also found 
that BIE did not conduct formal data reliability checks on the performance 
data that grantees report for the Native Language Immersion Cooperative 
Agreement grant program. Officials explained in October 2019 that 
although they pair grantee performance reports with conference calls to 
check on the work grantees are doing and reporting on, due to limited 
information system and data management capacity, they could not 
conduct formal data reliability checks on grantee performance data. As of 
April 2020, BIE has taken steps to increase its data management 
capacity. Specifically, a BIE official stated that the bureau had appointed 
him as chief performance officer and hired two additional officials with 
data system expertise in February 2020 to staff a performance office. 
Among the responsibilities of the BIE performance office, according to 
this official, are managing a bureau-wide data governance board, 
reviewing an inventory of bureau data systems, and ensuring the integrity 
of these systems. This official also stated that he anticipates that the BIE 
performance office will assist the office that administers the Native 
Language Immersion Cooperative Agreement in identifying performance 
metrics and designing an effective system of data collection, 
management, and analysis. 


A leading practice we identified for performance measurement states that 
agencies should assess grantee performance data for quality and 
reliability. As previously mentioned, assessing data reliability could 


61The recommendations described in the 2019 High Risk Report for BIE relate to, among 
other things, addressing its capacity issues. For example, we reported that the bureau had 
not yet completed a workforce plan to address staffing and training gaps, which we 
previously recommended and Interior's Office of the Assistant Secretary for Indian Affairs 
concurred with. See GAO-19-157SP, 129-130. 
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include activities such as asking selected questions to grantees or 
requesting documentation from them about how they collect data or 
grantees’ opinion of the quality of the data they are providing, and then 
reviewing these responses and documents. As the BIE performance 
office continues enhancing the bureau’s data management capacity, 
developing a process to assess the reliability of a sample of grantee 
performance data for the Native Language Immersion Cooperative 
Agreement could help BIE better understand how the program and its 
grantees are performing. 


Tribal Officials Cited 
Certain Risk Factors 
and Difficulties 
Implementing 
Programs among 
Challenges to 
Preventing or 
Addressing Juvenile 
Delinquency 


Tribal Officials Cited 
Substance Misuse and 
Educational Challenges 
among the Risk Factors 
That Contribute to 
Juvenile Delinquency in 
Their Communities 


Officials from all 12 interviews we conducted with tribes or tribal consortia 
(tribal officials) cited risk factors that contribute to juvenile delinquency in 

their communities, such as substance misuse, difficulties youth encounter 
in the educational system, and mental health issues. See figure 6 below 
for more information on the most common risk factors tribal officials cited. 


62As mentioned earlier, an OJJDP literature review identified risk factors for tribal youth 
that could contribute to a higher likelihood of contact with the criminal justice system, 
including historical trauma, violence, suicide, substance misuse, and lack of cultural 
instruction. 
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Substance misuse 


Educational challenges 


Figure 6: Number of Interviews in Which Officials from Selected Tribes or Tribal 
Consortia Cited Various Risk Factors That Contribute to Juvenile Delinquency 


Risk factor for juvenile delinquency 


Substance misuse 


Educational challenges 


Mental health 


Interpersonal violence 


Poverty 


Historical trauma 


0 1 2 3 4 5 6 7 8 9 10 11 12 
Number of interviews 


Source: GAO analysis of interviews with officials from federally recognized tribes and tribal consortia. | GAO-20-600 


Note: The figure includes the most common risk factors for juvenile delinquency described by tribal 
Officials. 


Tribal officials in 11 of the 12 interviews said that substance misuse was a 
contributing factor to juvenile delinquency in their communities. They cited 
underage alcohol use, marijuana use, and opioid use as examples of 
substance misuse. For example, an official from a tribal consortium noted 
that marijuana use was a major juvenile delinquency issue. According to 
the official, state legalization of medical marijuana made the drug easier 
for youth to access and led to more widespread usage among youth. 
Additionally, officials from a tribe in Minnesota noted that the opioid 
epidemic created instability for youth, as their caretakers or parents were 
absent from the home due to arrest or being in a substance abuse 
treatment program. 


Tribal officials in 10 of the 12 interviews also told us that Native American 
youth encounter challenges in the educational system and cited examples 
such as truancy, bullying, and lack of culturally sensitive instruction within 
their schools. For example, an official from a tribe in Minnesota 
mentioned truancy as the most pressing of the educational challenges 
facing his community. The official noted that over 50 percent of students 
in the reservation’s school district had truancy issues. In addition, an 
official from a tribe in California told us that many youth end up back in 
the justice system because they do not want to admit that they cannot 
read or write and, therefore, cannot complete required parole or probation 


Page 35 GAO-20-600 Native American Youth 


Mental health 


Tribal Officials Cited 
Conditions or Restrictions 
on Program Funding and 
Difficulties Communicating 
with Agencies among the 
Barriers to Implementing 
Federal Programs 


programs. The tribe has addressed this issue by enabling these youth to 
complete assignments using dictation programs on their phones. 


Tribal officials in six of the 12 interviews cited mental health issues, which 
included limited access to mental health services and providers, 
undiagnosed mental illness, and high suicide rates. For example, an 
official from a tribe in California mentioned that a survey of middle and 
high school students in her community showed that 30 percent of 
respondents had contemplated suicide. In addition, officials from a tribe in 
Minnesota said that in remote areas it is difficult to hire and retain mental 
health providers and for individuals to get a diagnostic screening. 
Furthermore, an official participating in a discussion group of DOJ 
Coordinated Tribal Assistance Solicitation Purpose Area 9: Tribal Youth 
Program grantees in California (California discussion group) noted the 
difficulty of connecting youth to behavioral health services and helping 
family members recognize the signs of mental illness. 


Tribal officials in 11 of the 12 interviews we conducted cited barriers to 
implementing federal programs, such as conditions or restrictions on how 
tribes may use program funding and communication challenges with 
agencies as well as hiring and retaining qualified program staff.63 See 
figure 7 below for more details on the specific challenges in implementing 
federal programs that officials cited in our interviews. 


63In GAO-18-591, we discussed challenges tribal officials face when applying for federal 
grant programs, including short applications windows, trouble collecting data for grant 
applications, and difficulty hiring grant writers. In this report, we focus on challenges tribal 
officials face implementing federal grant programs. 
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Conditions or restrictions on 
program funding 


Figure 7: Number of Interviews in Which Officials from Selected Tribes or Tribal 
Consortia Cited Various Challenges Implementing Federal Programs 


Type of implementation challenge 
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program funding 
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Source: GAO analysis of interviews with officials from federally recognized tribes and tribal consortia. | GAO-20-600 


Note: The figure includes the most common challenges implementing federal programs described by 
tribal officials. 


Tribal officials from nine of the 12 interviews noted that the federal 
government places conditions or restrictions on the use of federal 
funding. For example, officials from a tribe in Oklahoma noted that federal 
limitations on using certain funding streams for religious practices hinders 
their ability to provide culturally based interventions in therapeutic 
settings.® Likewise, these officials stated that restrictions on how 
disparate federal funding streams can be combined prevent treatment 
facilities from providing wraparound social services to address complex 


64In general, entities that receive federal funding may not use funds for explicitly religious 
activities such as worship, prayer, and proselytizing, and services offered by these entities 
must be provided to qualified beneficiaries regardless of the religious or nonreligious belief 
of the individuals. See, e.g., 28 C.F.R. § 38.5; 34 C.F.R. § 76.52; 42 C.F.R. § 54a.4; 45 
C.F.R. § 87.3. However, the American Indian Religious Freedom Act provides that federal 
policy is to protect and preserve for American Indians their right to exercise their religion. 
See 42 U.S.C. § 1996. Education officials reported that, pursuant to this policy, agencies 
generally are to consider their policies and procedures with the aim of protecting Indian 
religious freedom. 
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Communication with program 
Staff 


Hiring and retaining staff 


behavioral health issues.®5 Moreover, officials from another tribal 
consortium felt that federal funding levels are too static to meet the 
increasing demand for justice services resulting from growing arrest rates 
in their communities. Finally, an official who participated in a discussion 
group in New Mexico at a child welfare conference (New Mexico 
discussion group) told us that not being able to provide food for events 
when using federal funding makes it challenging to generate 
attendance. 


Tribal officials in seven of the 12 interviews cited communication 
challenges with federal officials as a barrier to implementing federal 
programs. Specifically, these officials mentioned communication 
challenges, such as late or infrequent responses from federal officials as 
well as difficulty using web-based platforms to upload performance data 
and supporting documentation due to poor internet access. For example, 
an official from a tribe in Alaska noted that her tribe’s DOJ program 
manager for one of its grants was either not available or accessible to 
answer questions, prompting the tribe to draw upon former DOJ program 
managers they had experience with for advice and support. 


According to tribal officials in seven of the 12 interviews, difficulty hiring 
and retaining program staff posed a barrier to implementing federal 
programs. Officials from one tribal consortium noted difficulty finding 
social workers who can provide support services for youth in rural areas. 
Officials from a tribe in Oklahoma noted the difficulty of hiring the staff— 
ranging from corrections officers to social workers—needed to provide 
wraparound social services that could address juvenile delinquency. 
Similarly, officials from a tribe in New Mexico told us that job vacancies 
for their federally funded programs can sometimes remain unfilled for 
years. Furthermore, an official who participated in the New Mexico 
discussion group told us that her tribe needs to hire non-Native American 
experts when implementing grant programs due to a lack of qualified 
Native American experts. She said this can be a challenge because non- 


65According to OJJDP, “wraparound is a youth-guided, family driven team planning 
process that provides coordinated and individualized community-based services for 
youths and their families to help them achieve positive outcomes... rather than forcing 
them to enroll in pre-determined, inflexible treatment programs.” Office of Juvenile Justice 
and Delinquency Prevention, Literature Review: A Product of the Model Programs 
Guide—Wraparound Process (Washington, D.C.: last updated April 2014). 


86AIthough federal funds are generally not available to pay for food, food purchases for 
events may be permitted under certain circumstances. 
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Positive experiences with 
federal programs 


Tribal Officials Cited 
Justice System-Related 
Challenges as Barriers to 
Preventing or Addressing 
Juvenile Delinquency 


Native American experts may lack cultural sensitivity and be reluctant to 
assimilate to the tribe’s culture.6” We also asked officials in the New 
Mexico discussion group how many of their tribes employed a dedicated 
grant writer, and only two out of 17 participants indicated they did. 


Tribal officials in seven of the 12 interviews also mentioned positive 
experiences in implementing federal programs. For example, officials 
from a tribe in Minnesota said that as a result of the tribe’s strong working 
relationship with their program manager from Interior's Bureau of Indian 
Affairs, they had successfully used funding to build upon their tribe’s 
juvenile justice data infrastructure. Moreover, officials from a tribe in New 
Mexico mentioned the positive experiences they had working with their 
program managers for the HHS Indian Health Service’s 
Methamphetamine and Suicide Prevention Initiative grant program and 
generally felt that the federal funding they had received was helping to 
prevent juvenile delinquency. Finally, officials from a tribal consortium in 
Oklahoma said they had overwhelmingly positive and consistent support 
from DOJ program staff as part of an OJJDP grant program. Officials from 
one tribe in the consortium used the OJJDP grant to develop a Juvenile 
Healing to Wellness Court. Wellness courts are special court dockets that 
can combine judicial supervision, substance abuse treatment, case 
management, and drug testing, along with incentives and sanctions to 
help individuals with substance use disorders. 


In all 12 interviews that we conducted, tribal officials cited justice system- 
related challenges, which included coordination and jurisdictional issues, 
difficulty collecting accurate criminal justice and other types of data in 
their communities, and law enforcement-related concerns. See figure 8 
below for more information on justice-system-related challenges officials 
cited. 


87The Indian Self-Determination and Education Assistance Act, as amended, requires that 
federal grants to Indian organizations or grants for the benefit of Indians must require to 
the greatest extent feasible preferences and opportunities for training and employment in 
connection with administering the grant to be given to members of federally recognized 
tribes. 25 U.S.C. § 5307(b)(1). 
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Coordination and jurisdictional 
issues 


Figure 8: Number of Interviews in Which Officials from Selected Tribes or Tribal 
Consortia Cited Various Justice System-Related Challenges to Preventing or 
Addressing Juvenile Delinquency 
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Note: The figure includes the most common justice system-related challenges described by tribal 
Officials. 


Tribal officials in seven of the 12 interviews cited coordination and 
jurisdictional issues such as difficulties interacting with state and local 
justice systems. For example, officials from a tribe in New Mexico told us 
that despite the state’s tribal notification process that requires nontribal 
law enforcement to identify Native American children taken into custody, 
the tribe had not been notified of an ongoing criminal case involving a 
child from their tribe.®8 These officials also noted that they have had to 
arrange regular meetings with state juvenile justice officials to ensure that 
the notification requirements were followed. Officials from a California 
tribe also told us that the tribe’s relationship with local law enforcement 
has experienced challenges in recent years. The official indicated that 
although state and local law enforcement agencies have jurisdiction over 
major crimes in Indian country in California, the tribe wants outside law 
enforcement to better recognize and respect its sovereignty. ® 


68New Mexico law requires nontribal law enforcement to identify Native American children 
taken into custody and contact the children’s tribes in delinquency cases. N.M. Stat. § 
32A-2-5. 


69Public Law 280 gave certain states—Alaska, California, Minnesota, Nebraska, Oregon, 
and Wisconsin—criminal jurisdiction over offenses committed by or against Indians in 
Indian country, except as specified in statute, and renounced federal jurisdiction in Indian 
country in those states. 18 U.S.C. § 1162. 
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Data collection 


Law enforcement 


Tribal officials in six of the 12 interviews we conducted noted the difficulty 
of collecting justice-system-related data. For example, an official from a 
tribe in California mentioned that the state of California does not 
document the Native American status or tribal affiliation of juveniles 
during intake.”° This official added that tracking the tribe’s youth 
population in the California State juvenile justice system requires 
manually checking two databases and relying on name recognition, 
making it difficult for tribal officials to identify juvenile delinquency trends 
in their communities. Officials who participated in the New Mexico 
discussion group also mentioned data collection concerns. For example, 
one official stated that sharing data about tribal communities is difficult 
because of stipulations within the Health Insurance Portability and 
Accountability Act (HIPAA), which requires the confidential handling of 
protected health care information. The official added that tribes are 
starting to work on their internal laws regarding sharing data about their 
citizens. Another official who participated in the New Mexico discussion 
group acknowledged the need to improve data about tribal communities 
but said that federal agencies need to do a better job of listening to tribes, 
who are the owners of the data. 


Tribal officials in six of the 12 interviews cited law enforcement 
challenges, specifically resource constraints on tribal law enforcement, 
difficulty establishing positive relationships with nontribal law enforcement 
agencies, and difficulty training and retaining tribal law enforcement 
officers. For example, officials from a tribe in New Mexico noted their 
community’s difficulty in providing the resources to sustain an effective 
police force and mentioned that the tribe had only four police cars but did 
not have the proper equipment to operate them. In addition, officials 
representing a tribal consortium said it was difficult to retain officers with 
special certifications, such as canine and drug specialties, because 
certified officers often leave for better-paid positions elsewhere. 


70According to OJJDP, intake is the process by which the juvenile probation department or 
the prosecutor's office, when receiving a juvenile delinquency case referral, decides to 
dismiss the case, handle the matter informally, or request formal intervention by the 
juvenile court. 
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Tribal Officials Most Often 
Cited Ability to Implement 
Culturally Relevant 
Programs as a Promising 
Practice for Preventing or 
Addressing Juvenile 
Delinquency 


Officials in all 12 interviews we conducted with tribes or tribal consortia 
cited promising practices for preventing or addressing juvenile 
delinquency. Specifically, tribal officials in eight of the 12 interviews cited 
culturally relevant programs as being a promising practice for preventing 
or addressing juvenile delinquency in their communities. According to 
these officials, culturally relevant programs can consist of traditional 
spiritual, linguistic, and artistic practices and reflect tribal values around 
intergenerational connection and family engagement. Officials from a tribe 
in New Mexico said they use a prevention approach with these types of 
programs. They said, while not directly focused on delinquency, these 
programs allow children to get away from problems at home and 
strengthen their cultural identity. In addition, an official from a tribe in 
Alaska noted the success of her community's cultural summer camp, 
which gave youth the opportunity to engage in traditional music, arts, and 
dance activities and connect with their tribal identity. Finally, an official in 
the California discussion group noted the importance of incorporating 
traditional beliefs and experiences in all their youth programming, such as 
offering bilingual and bicultural rites of passage. 


In addition, officials from tribes in New Mexico and California described 
community interventions for juvenile offenders that incorporated 
traditional values around justice and mediation. For example, a tribe in 
New Mexico has historically had numerous “informal” sheriffs that conduct 
interventions with parents in cases of juvenile delinquency. Overall, 
officials said the tribe’s law enforcement officials emphasize community 
policing, such as by interacting with youth outside the justice system 
through shopping trips or riding go-carts. Officials from a tribe in California 
used funding from the OJJDP Juvenile Healing to Wellness Court grant 
program to provide a space to focus on restorative, rather than punitive, 
justice. In the tribe’s wellness court, children meet with a jury of their 
peers composed of youth who have already completed the program. The 
jury listens to cases and chooses sentences from a range of options, 
which could include solutions such as an apology letter, raising their 
grades, or community service. Officials said an important part of the 
program includes having a follow-up officer to ensure that the youth meet 
the terms of their sentences. 


Officials in seven of the 12 interviews we conducted also noted partnering 
with state and local stakeholders, such as nontribal justice systems, state 
governments, and public schools in their communities, as a promising 
practice. For example, a California tribal official told us that her tribe 
successfully established a memorandum of understanding with their 
nearest county court system, which allowed tribal officers to be trained 
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Conclusions 


and serve in county courts. In addition, officials from a tribal consortium 
noted they are partnering with a state government to create an intertribal 
committee to address tribal juvenile justice issues in the region, 
particularly in the area of data collection. 


Some Native American youth face challenges such as historical trauma, 
exposure to violence, substance abuse, and high poverty rates, which, 
along with tribal communities’ lack of funding for mental health and other 
services, make them susceptible to becoming involved with the justice 
system. HHS, DOJ, Education, and Interior administered grant programs 
targeted to tribes from fiscal years 2015 through 2018 that could prevent 
or address juvenile delinquency. HHS administers grant programs that 
target challenges that some Native American youth face, such as 
substance misuse and limited access to child and family behavioral health 
specialists. 


While these agencies incorporated almost all leading practices we 
identified for performance measurement or program evaluation of grant 
programs, we identified opportunities for improvement that could help 
them better assess the performance of selected programs. In particular, 
HHS’s Administration for Children and Families does not assess the 
reliability of grantee performance data contained within annual 
performance reports that tribal recipients submit for the Stephanie Tubbs 
Jones Child Welfare Services Program. While assessing the reliability of 
tribal grantee performance data is not a requirement for this program, 
developing a process to do so could help provide the Administration for 
Children and Families further assurance that it has an accurate 
representation of tribal grantee performance and improve its ability to 
identify any potential programmatic challenges that might emerge. 


Since fiscal year 2017, Interior's Bureau of Indian Education has not 
consistently alerted some grantees of the Native Language Immersion 
Cooperative Agreement when they are late in submitting performance 
reports. Taking steps to alert grantees of the program when they are late 
in submitting performance reports could help the bureau ensure that 
grantees are complying with program requirements. This could also 
position the bureau to respond in a timely manner to any performance 
issues during the life cycle of the grant program as it continues to address 
its staffing challenges. The bureau could also better understand how 
grantees of the Native Language Immersion Cooperative Agreement are 
performing by developing a process to assess the reliability of a sample 
of grantee performance data. 
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Recommendations for 
Executive Action 


Agency Comment 
and Our Evaluation 


We are making three recommendations, including one to HHS and two to 
Interior's Bureau of Indian Education. 


The Associate Commissioner of the Administration for Children and 
Families Children’s Bureau within the Department of Health and Human 
Services should develop a process to assess the reliability of grantee 
performance information contained within annual performance reports for 
tribal recipients of its Stephanie Tubbs Jones Child Welfare Services 
Program. (Recommendation 1) 


The Director of the Department of the Interior's Bureau of Indian 
Education should take steps to alert grantees of the Native Language 
Immersion Cooperative Agreement when they are late in submitting 
performance reports. (Recommendation 2) 


The Director of the Department of the Interior's Bureau of Indian 
Education should develop a process to assess the quality and reliability of 
a sample of grantee performance data for the Native Language 
Immersion Cooperative Agreement. (Recommendation 3) 


We provided a draft of this product to DOJ, HHS, Interior, and Education 
for review and comment. DOJ told us that they had no comments on the 
draft report. We received comments from HHS, which we have 
reproduced in appendix IV. In its comments, HHS disagreed with our 
recommendation. We also received comments from Interior, which we 
have reproduced in appendix V. In its comments, Interior agreed with the 
two recommendations we made to the agency and described actions 
planned to address them. Additionally, we received comments from 
Education, which we have reproduced in appendix VI. In its comments, 
Education expressed two areas of concerns with the draft product. 
Education also provided technical comments, which we incorporated as 
appropriate. 


In its comments, HHS stated that it did not concur with our 
recommendation. Specifically, HHS stated that statutorily mandated 
performance measurement requirements for the Stephanie Tubbs Child 
Welfare Services Program do not apply to tribal recipients. As we noted in 
the report, officials from the HHS ACF Children’s Bureau told us that they 
do not attempt to independently validate the reliability of tribal grantee 
performance data for the Stephanie Tubbs Jones Child Welfare Services 
Program because they are not required to do so and because funding 
decisions for grantees are not tied to performance metrics. However, we 
maintain that assessing the reliability of grantee performance information 
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from tribal recipients could help provide the ACF Children’s Bureau 
further assurance that it has an accurate representation of grantee 
performance. 


HHS also stated that the ACF Children’s Bureau’s joint planning process 
with tribes to develop a plan to implement the Stephanie Tubbs Jones 
Child Welfare Services Program is sufficient to assess the reliability of 
grantee performance information and that any additional activities would 
be overly burdensome and resource intensive. Pursuant to federal 
regulations, the joint planning process is to involve discussions, 
consultation, and negotiation between ACF and the tribes in areas such 
as identifying needs and developing goals and objectives that will result in 
improving outcomes.” 


We acknowledge HHS’s statement that this joint process may be used to 
assess the reliability of any grantee performance information, generally. 
However, assessing grantee performance information for tribal recipients 
contained within annual performance reports, specifically, is important 
since these reports are where tribal recipients provide updates on the 
progress they made toward the goals and objectives set forth during the 
joint planning process. Therefore, we continue to believe that the 
recommendation is warranted. However, we modified the 
recommendation to clarify that the ACF Children’s Bureau should assess 
the grantee performance information that is contained within their annual 
performance reports for tribal recipients of the Stephanie Tubbs Jones 
Child Welfare Services Program. Recognizing HHS’s resources concerns, 
we maintain that asking tribal recipients to describe how they collect data 
or obtaining their opinions on the quality of the data in their annual 
performance reports and then reviewing these responses could be a cost- 
effective way to assess performance information. Developing a process to 
assess the reliability of performance information in the annual reports 
could help HHS better understand the program and grantee performance. 


Regarding Interior, the agency concurred with our recommendation that 
BIE should take steps to alert grantees of the Native Language 
Immersion Cooperative Agreement when they are late in submitting 
performance reports. In its comments, Interior noted that BIE, in general, 
and the office that administers the program, specifically, faces human 
capital challenges. However, Interior stated that as the Native Language 
Immersion Cooperative Agreement matures and the office that 


7145 C.F.R. § 1357.10(c). 
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administers the program fills critical positions, BIE anticipates that the 
issue of timely and regular performance reporting will be addressed. In 
our report, we noted that BIE faces ongoing staffing gaps. Even as the 
program is maturing, we maintain that taking steps to alert Native 
Language Immersion Cooperative Agreement grantees when they are 
late in submitting performance reports could help the bureau ensure that 
grantees are complying with the terms and conditions of the grant 
program. Doing so could also help BIE officials respond in a timely 
manner to any performance issues during the life cycle of the grant 
program. 


Regarding our second recommendation to Interior, the agency concurred 
that BIE should develop a process to assess the quality and reliability of a 
sample of grantee performance data for the Native Language Immersion 
Cooperative Agreement. In its comments, Interior stated that the BIE 
performance office will be tasked with assisting the office that administers 
the program in inventorying, reviewing, and improving data collection and 
analysis. Additionally, Interior stated that, as part of this work of improving 
data collection and analysis, BIE anticipates that the Native Language 
Immersion Cooperative Agreement program will develop a process to 
assess the quality and reliability of data. If this process is implemented 
effectively, we believe that these actions would be positive steps toward 
addressing our recommendation. To fully address our recommendation, it 
will be important for BIE to take steps to develop a process to assess the 
reliability of a sample of grantee performance data for the Native 
Language Immersion Cooperative Agreement even as the BIE 
performance office continues to enhance the bureau’s data management 
capacity. 


Regarding Education, the agency raised two areas of concern in its 
comments. First, Education noted that the use of the term “delinquency” 
in the report has pejorative implications and does not appear appropriate 
when referencing programs that are not intended to address delinquency. 
We note that our use of the term “delinquency” in reference to grant 
programs that could prevent or address delinquency is consistent with our 
September 2018 report on Native American youth involvement in the 
justice system.72 In our report, we selected grant programs (1) whose 
summary descriptions or purposes explicitly mentioned tribes or Native 
Americans as beneficiaries and (2) whose activities related to the risk or 
protective factors discussed in the OJJDP Tribal Youth in the Juvenile 


72For example, see GAO-18-591, 2. 
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Justice System literature review, among other criteria, as relevant for this 
review. Similarly, the Juvenile Justice and Delinquency Prevention Act of 
1974, as amended, defines “juvenile delinquency program”, in part, as 
“any program or activity designed to reduce known risk factors for juvenile 
delinquent behavior, provides activities that build on protective factors for, 
and develop competencies in, juveniles to prevent, and reduce the rate 
of, delinquent juvenile behavior’. 75 


Second, Education expressed concern with the use of the phrase “Grant 
Programs Addressing Delinquency” in the title of our draft report because 
it believed that such a title implies that the purpose of each of the 
programs listed in the report is to address the topic of delinquency. 
Education stated that addressing delinquency is not the purpose of its 
grant programs and the agency’s mission, generally. Instead, Education 
stated that the purpose of its grant programs and its agency mission is to 
improve educational opportunities, access, and equity for all children, 
including Native American students. In response, we modified the title of 
the report to emphasize our focus on federal grant programs that could 
prevent or address delinquency. We also added a statement in the report 
noting that while the Education programs cited in the report could 
potentially reduce student interactions with the justice system, doing so is 
not the statutory purpose of these programs. 


We are sending copies of this report to the appropriate congressional 
committees, the Attorney General, Secretary of Health and Human 
Services, Secretary of the Interior, and Secretary of Education, as well as 
other interested parties. In addition, the report will be available at no 
charge on the GAO website at http://www.gao.gov. 


73See 34 U.S.C. § 11103(3). 
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If you or your staff have any questions about this report, please contact 
me at (202) 512-8777 or goodwing@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made key contributions to this 
report are listed in appendix VII. 


Gretta L. Goodwin 
Director, Homeland Security and Justice 
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Appendix |: Objectives, Scope, and 
Methodology 


This report addresses (1) the extent to which federal agencies provided 
financial assistance targeted to tribes that could help prevent or address 
delinquency among American Indian or Native Alaskan (Native American) 
youth; (2) the extent to which federal agencies assess the performance of 
selected grant programs that could help prevent or address delinquency 
among Native American youth and incorporate leading practices we 
identified when doing so; and (3) delinquency issues selected tribes 
report encountering within their communities and the types of challenges 
they face, if any, in preventing or addressing these issues. 


To address the first objective, we identified relevant federal financial 
assistance available from fiscal years 2015 through 2018,1 which included 
discretionary grants and cooperative agreements as well as formula 
grants (collectively referred to as grant programs).2 To identify these grant 
programs, an analyst conducted a keyword search of “youth or juvenile” 
in Grants.gov—an online repository that houses information on over 
1,000 different programs across federal agencies—as well as agencies’ 
websites. With the Grants.gov keyword search, the analyst specifically 
selected results within the following federal agencies that administered 
the overwhelming majority of recent funding serving Native American 
communities: the Departments of Agriculture; Education (Education); 
Health and Human Services (HHS); Housing and Urban Development; 
Justice (DOJ); Labor; the Interior (Interior); Transportation; and the 
Environmental Protection Agency.? 


The analyst then selected discretionary grant programs in the search 
results whose summary descriptions on Grants.gov mentioned tribal or 


1We selected this period to focus on the most recent trends in available funding from 
when we began the review. 


2Discretionary grants are competitive in nature, whereby the granting agency has 
discretion to choose one applicant over another based on eligibility and merit. Cooperative 
agreements are similar to discretionary grants in that federal agencies generally award 
them to grantees competitively based on eligibility and merit. In contrast to discretionary 
grants though, federal agencies generally use cooperative agreements when they 
anticipate that there will be substantial federal involvement with the recipient during the 
performance of the program activities. According to Grants.gov, a formula grant is a type 
of grant that grantors award to grantees based on statistical criteria. Authorizing legislation 
and regulations usually define the statistical criteria and the funding amount allocated to 
recipients of these types of grants. 


3For example, the fiscal year 2019 Federal Funding for Program Serving Tribes and 
Native American Communities, which shows that federal agencies enacted approximately 
$22.017 billion for programs that served Native Americans in fiscal year 2018, and that 
approximately $21.433 billion, or about 97 percent, came from the nine federal agencies 
listed above. 
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Methodology 


Native American youth as beneficiaries and were related to the following 
categories: 

e Child separation from home; 

e Cultural values and institutions; 

e Cross-jurisdictional cooperation and information sharing; 

e Exposure to violence; 

e Family; 

e Historical or intergenerational trauma; 

e Mental health and trauma-informed services; 

e Poverty; 

e Reforms and alternatives to detention; 

e Resources to enhance or reform the tribal juvenile justice system; 

e Suicide; 

e Substance use and abuse; and 

e Miscellaneous (for example, running away from home, etc.).4 

Two analysts collaborated to develop these categories. Specifically, one 
analyst reviewed selected reports focused on Native American juvenile 
justice and then identified risk factors, protective factors, and 


recommendations related to tribal juvenile delinquency. A second analyst 
either confirmed agreement with the categories or discussed any 


4We excluded grant programs that focused on providing services for victims of crimes and 
violence from this review, just as we did in GAO, Native American Youth: Involvement in 
Justice Systems and Information on Grants to Help Address Juvenile Delinquency, 
GAO-18-591 (Washington, D.C.: Sept. 5, 2018). We also excluded selected Interior 
National Park Service discretionary cooperative agreements that facilitated Native 
American youth involvement in tribal cultural resource projects but did not accept 
applications from other entities. 
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Methodology 


differences to reach a consensus.® The reports the first analyst reviewed 


were 


e DOJ Office of Juvenile Justice and Delinquency Prevention (OJJDP) 
Tribal Youth in the Juvenile Justice System literature review; 


e “Creating a Juvenile Justice System that Focuses on Prevention, 
Treatment, and Healing”; Chapter 4 of the November 2014 Attorney 
General’s Advisory Committee on American Indian/Alaska Native 
Children Exposed to Violence report, Ending Violence so Children 
Can Thrive report;® and 


e “Juvenile Justice: Failing the Next Generation,”; Chapter 6 of the 
November 2013 Indian Law and Order Commission Report to the 
President and Congress of the United States, A Roadmap for Making 
Native America Safer report.” 


Regarding formula grant programs, the first analyst selected those 
programs whose summary descriptions mentioned youth as beneficiaries, 
related to the aforementioned categories, and allowed federally 
recognized tribes (tribes) or tribal organizations to apply to receive 


5This literature review identified risk factors for tribal youth, which are characteristics or 
activities that could contribute to a higher likelihood of their contact with the criminal justice 
system. These risk factors included historical trauma, violence, suicide, substance use, 
and lack of cultural instruction. The literature review also listed various protective factors— 
characteristics of the child, family, and wider environment that can increase resiliency and 
reduce the likelihood of negative child outcomes and behaviors, such as contact with the 
juvenile justice system. The protective factors in the literature review included family and 
culture. See Office of Juvenile Justice and Delinquency Prevention, Literature Review: A 
Product of the Model Programs Guide—Tribal Youth in the Juvenile Justice System 
(Washington, D.C.: last updated April 2016). 


8The Attorney General’s Advisory Committee on American Indian/Alaska Native Children 
Exposed to Violence (Advisory Committee), Ending Violence so Children Can Thrive 
(Washington, D.C.: November 2014). 


7Indian Law and Order Commission Report to the President and Congress of the United 
States, A Roadmap for Making Native America Safer (November 2013). 
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Methodology 


funding directly. A second analyst then reviewed the first analyst’s initial 
determinations of potentially relevant grant programs and either 
confirmed agreement or discussed any differences of opinion to reach a 
consensus. 


After developing an initial list of grant programs based on our review of 
Grants.gov search results as well as agency websites, we shared the list 
with the agencies from which we initially identified relevant programs— 
HHS, DOJ, Education, and Interior—to obtain their perspectives. Despite 
these steps, it is possible that we did not identify all potentially relevant 
federal grant programs targeted to tribes that could prevent or address 
delinquency among Native American youth. Moreover, for some grant 
programs we identified, addressing juvenile delinquency is not the 
primary purpose and beneficiaries may include youth who are neither 
Native American nor have a history of delinquency. We included them as 
programs that could address juvenile delinquency because, as mentioned 
above, their described activities relate to the risk or protective factors 
discussed in the OJJDP Tribal Youth in the Juvenile Justice System 
literature review, among other criteria. 


After finalizing the list of relevant grant programs, we obtained fiscal year 
2015 through 2018 award data for these programs. We assessed the 
reliability of the award data by reviewing related documentation and 
performing checks for missing data and other obvious errors. We also 
asked knowledgeable officials about the information systems in which 
their agencies maintain the data; the extent to which their agencies 
conduct reviews to ensure that the data are accurate; and whether there 
are any known limitations to the data, among other questions. We 
determined that the award data were sufficiently reliable for the purposes 
of reporting funding trends for grant programs that could have helped 


8For example, we selected the Department of Health and Human Services’ Promoting 
Safe and Stable Families formula grant program as relevant for our review because its 
primary goals include preventing the unnecessary separation of children from their 
families, which relates to one of the categories we identified for tribal juvenile 
delinquency—child separation from home. We also selected this program because it 
reserves 3 percent of its total annual funding for eligible tribes and tribal consortia for 
which tribal entities can apply to receive directly. In fiscal year 2017, for example, total 
funding for the Promoting Safe and Stable Families program was about $381 million, and 
tribal entities received approximately $11.5 million, or about 3 percent, of the program’s 
total funding. In contrast, the DOJ Title Il Formula Grants Program provides funding to 
states to develop programs to address delinquency and improve the juvenile justice 
system. However, we did not ultimately select this program as relevant for our review 
because tribal entities are not eligible to apply directly to receive funding. 
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prevent or address delinquency among Native American youth during 
fiscal years 2015 through 2018. 


In addition to identifying relevant grant programs for the first objective, we 
also interviewed agency officials from Interior and HHS’s Indian Health 
Service that administer self-determination contracts and self-governance 
compacts with Indian tribes (tribes). We conducted these interviews to 
determine the extent to which the agreements may include financial 
assistance that tribes might use to prevent or address delinquency among 
Native American youth. Authorized by the Indian Self-Determination and 
Education Assistance Act of 1975, as amended, self-determination 
contracts and self-governance compacts allow tribes (or tribal 
organizations and consortia) to enter into agreements with Interior and 
HHS’s Indian Health Service, which enable them to assume the 
administration of certain federal programs—or portions of them—that the 
agencies themselves previously managed. 2 


To address our second objective, we requested performance- 
measurement-related documentation, such as performance reports from 
one to two grantees of a nonprobability sample of 15 HHS, DOJ, 
Education, and Interior grant programs that we identified within our first 
objective. 19 In selecting this sample of grant programs, we employed a 
multistage process that considered (1) the federal agency that 
administers the program, (2) the agency component office that manages 
the program, (3) and the program type (that is, discretionary versus 
formula grant program). In considering these three factors, we were able 
to select a sample that represented a mix of agencies, component offices, 
and types of programs. Furthermore, we requested program evaluation 
reports published from fiscal years 2015 through 2018 that involved or 
focused on Native American youth regardless of the program. We also 
conducted interviews with or submitted questions to officials from HHS, 
DOJ, Education, and Interior. For example, we asked cognizant agency 


%Although both types of agreements transfer the administration of federal programs to 
tribes or tribal organizations, there are differences between the agreements. In January 
2019, we reported on differences between the self-determination contracts and self- 
governance compacts that the Department of the Interior enters into with tribes and tribal 
organizations. See GAO, Indian Programs: Interior Should Address Factors Hindering 
Tribal Administration of Federal Programs, GAO-19-87 (Washington, D.C.: Jan. 3, 2019), 
10. 


10For the purposes of this report, assessing performance refers to agencies determining 
the extent to which a grantee or grant program has made progress on its goals or how 
well a grant program is working, as measured by the incorporation of leading practices we 
identified for performance measurement and program evaluation. 
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officials to describe how their agency or component offices measured the 
performance of their grantees and how frequently they did so. See table 5 
for a list of our selected sample of 15 grant programs. 


SS a a a ae ee ee a ES ee ES eS ee ee ee ee 
Table 5: List of Selected Departments of Health and Human Services, Justice, Education, and the Interior Grant Programs for 
Assessment against Leading Practices 


Name of grant program Department Component office 

Coordinated Tribal Assistance Solicitation - Purpose Justice Bureau of Justice Assistance 

Area 4: Tribal Justice System Infrastructure Program 

Coordinated Tribal Assistance Solicitation - Purpose Justice Office of Juvenile Justice and Delinquency 

Area 9: Tribal Youth Program Prevention 

Tribal Youth Program Training and Technical Assistance Justice Office of Juvenile Justice and Delinquency 
Prevention 

Native American Language (NAL@ED) Program Education Office of Elementary and Secondary Education 

Indian Education Formula Grants to Local Educational Education Office of Elementary and Secondary Education 

Agencies 

Native American and Alaska Native Children in School Education Office of English Language Acquisition 


Program 


Native American Language Preservation and 
Maintenance-Esther Martinez Immersion 


Health and Human 
Services 


Administration for Children and Families 


Stephanie Tubbs Jones Child Welfare Services 


Health and Human 
Services 


Administration for Children and Families 


Tribal Maternal, Infant, and Early Childhood Home 
Visiting Program: Development and Implementation 
Grants 


Methamphetamine and Suicide Prevention Initiative - 
Generation Indigenous (Gen-1) Initiative Support 


Health and Human 
Services 


Health and Human 
Services 


Administration for Children and Families in 
partnership with the Health Resources and 
Services Administration 


Indian Health Service 


Collaborative Hubs to Reduce the Burden of Suicide 


among American Indian and Alaska Native Youth (U19) 


Health and Human 
Services 


National Institutes of Health 


American Indian/Alaska Native Health Equity Initiative 


Health and Human 
Services 


Office of Minority Health 


Cooperative Agreements for State-Sponsored Youth 


Suicide Prevention and Early Intervention (PPHF-2015) 


Health and Human 
Services 


Substance Abuse and Mental Health Services 
Administration 


Cooperative Agreements for Tribal Behavioral Health 
(short title: Native Connections) 


Health and Human 
Services 


Substance Abuse and Mental Health Services 
Administration 


Native Language Immersion Cooperative Agreement 


Source: GAO. | GAO-20-600 


Interior 


Bureau of Indian Education 


In addition, we assessed the extent to which HHS, DOJ, Education, and 
Interior incorporated leading practices for performance measurement or 
program evaluation when assessing the performance of their grant 


Page 54 


GAO-20-600 Native American Youth 


Appendix |: Objectives, Scope, and 
Methodology 


programs.'1 To do this, we (1) identified leading practices for performance 
measurement and program evaluation and (2) determined the extent to 
which the four agencies incorporated these leading practices for the 
nonprobability sample of 15 programs we previously mentioned. 


To identify leading practices for performance measurement and program 
evaluation, we used a deductive content analysis approach. One analyst 
reviewed and coded passages of text that conformed to preestablished 
categories the analyst derived from a preliminary review of selected GAO 
and external sources (see full list of sources in app. II).12 The analyst then 
identified an initial set of leading practices from the coded passages of 
text by grouping the most common into two smaller subsets of leading 
practices, one each for performance measurement and program 
evaluation. A second analyst reviewed the initial leading practices to 
ensure that the two groups of leading practices were reasonable and 
comprehensive. After the first and second analyst reached consensus on 
the selected leading practices, we sent the practices to internal legal and 
methodological specialists for feedback. In total, we identified 14 leading 
practices for performance measurement and 14 leading practices for 
program evaluation, listed in tables 6 and 7 below. 


11See GAO, Best Practices Methodologies: A New Approach for Improving Government 
Operations, NSIAD-95-154 (Washington, D.C.: May 1, 1995). According to this report, 
best management practices refer to the processes, practices, and systems that are widely 
recognized as improving an organization’s performance and efficiency in specific areas. 
For the purpose of this review, the term “leading practices” refers to commonly mentioned 
and recommended practices related to performance measurement and program 
evaluation that we identified when reviewing GAO and external reports or guidance 
relevant to these topics. 


12The preestablished categories for performance measurement included the following: 
performance measurement planning and structure; performance goals and measures; 
data collection; reporting; use of performance results; and miscellaneous. For program 
evaluation, the preestablished categories were planning and structure; methodology; 
ethics and competency; reporting; use of evaluations; and miscellaneous. 
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Table 6: List of GAO-Identified Leading Practices for Performance Measurement 


Leading practices for performance measurement 


The agency should establish activities to measure performance that compare program achievements to program objectives. 


2. The agency should have performance plans that outline required performance measurement activities. 

3. The agency should incorporate risk assessment in its performance measurement activities. 

4. Performance measurement activities should be aligned with the needs of users of performance information. 
5. Management should use performance data when making programmatic and operational decisions. 

6. Results should be clearly communicated to users and stakeholders. 

7. Results should be reported periodically throughout a program’s life cycle. 

8. Performance goals and measures should cover core program activities and priorities. 

9. Performance goals and measures should align with agency- and/or government-wide priorities. 

10. Performance goals and indicators should be measurable, objective, and reliable. 

11. Performance goals and measures should clearly communicate performance targets. 

12. Performance measures should provide baseline data and measure program progress. 

13. Data should be assessed for quality and reliability. 

14. The agency should clearly communicate the scope and timing of performance measurement activities to grantees. 


Source: GAO. | GAO-20-600 


Note: The leading practices we identified for performance measurement are specific to GAO-20-600 
and do not necessarily reflect all possible leading practices in these areas. However, these leading 
practices are consistent with practices identified in our prior work and provide insights on how 
agencies could conduct performance measurement and program evaluation activities. 


Table 7: List of GAO-Identified Leading Practices for Program Evaluation 


Leading practices for program evaluation 


The agency should have evaluation plans. 


The agency should have written policies to guide evaluation activities and establish quality standards. 


Evaluation policies should require evaluators to have the necessary knowledge and competencies for the evaluation. 


Evaluation policies should have ethical behavior guidelines, including ensuring evaluator independence. 


Evaluation findings should be disseminated to stakeholders and the public, when possible. 
Management should use evaluation findings to make programmatic and operational decisions. 


Management should use evaluations to identify innovative or promising practices. 


Coy N} O) oy BR) wy) NM) > 


Evaluators should involve a full range of stakeholders in the planning process. 


co 


Evaluation should use the most rigorous methods appropriate to the questions and context. 


10. 


Evaluation questions should be feasible, relevant, clearly defined, and measureable. 


11. 


Data should be assessed for quality and reliability. 


12. The evaluation’s conclusions should be supported by the findings. 
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Leading practices for program evaluation 


13. The criteria and measures should be observable and appropriate to the evaluation questions. 


14. The evaluation report should include a discussion of the study’s limitations. 


Source: GAO. | GAO-20-600 


Note: The leading practices we identified for program evaluation are specific to GAO-20-600 and do 
not necessarily reflect all possible leading practices in these areas. However, these leading practices 
are consistent with practices identified in our prior work and provide insights on how agencies could 
conduct performance measurement and program evaluation activities. 


The leading practices we identified for performance measurement and 
program evaluation are specific to the engagement and do not 
necessarily reflect all possible leading practices in these areas. However, 
these leading practices are consistent with practices identified in our prior 
work and provide insights on how agencies could conduct performance 
measurement and program evaluation activities. 


To assess the extent to which agencies incorporated these leading 
practices, an analyst reviewed the performance measurement-related 
documentation for the nonprobability samples of grant programs and 
relevant program evaluation reports mentioned above and made a 
determination of whether the agency or component office incorporated 
the leading practices we identified. A second analyst then reviewed the 
first analyst's determinations and either confirmed agreement or 
discussed any differences of opinion to reach consensus. We did not 
assess the extent to which an agency incorporated leading practices for 
program evaluation if we could not identify an evaluation report the 
agency published between fiscal years 2015 and 2018 that focused on 
Native American youth.‘ We also did not include self-determination 
contracts and self-governance compacts that tribes have with Indian 
Health Service and Interior in the second objective. This is because 
monitoring of the implementation of these agreements is governed by the 
Indian Self-Determination and Education Assistance Act, as amended, 
and its implementing regulations, and monitoring and performance 
measurement standards that generally apply to federal grant programs do 
not generally apply to these agreements. Despite these steps, it is 
possible that we did not identify every program evaluation report that 
focused on Native American youth completed during this period. In 
addition, because we relied on a limited nonprobability sample of grant 


13We also excluded potentially relevant reports that, in our judgement, did not meet the 
definition of program evaluation contained within GAO, Performance Measurement and 
Evaluation: Definitions and Relationships, GAO-11-646SP (Washington, D.C.: May 2, 
2011). 
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programs, grantees, and program evaluation reports to determine the 
extent agencies incorporated the leading practices we identified, our 
findings are not generalizable to the universe of federal programs, 
grantees, and program evaluation reports. However, our findings do offer 
insights into agencies’ performance measurement and program 
evaluation practices, particularly as those practices relate to grant 
programs intended for tribal communities. Finally, although we evaluated 
the extent to which agencies incorporated leading practices for 
performance measurement and program evaluation, we did not assess 
the performance of any grant programs themselves. 


To address our third objective, we conducted 12 semistructured 
interviews with officials from 10 tribes and two tribal consortia.’4 We 
selected these interview participants to include tribes located near our site 
visit locations but also considered geographical diversity as well as a mix 
of smaller and larger tribes or tribal consortia. 15 


We identified common themes across the 12 interviews we conducted 
using NVivo, a qualitative analysis software program. Specifically, we 
conducted a preliminary review of the interview summaries to develop a 
list of possible barriers to preventing or addressing juvenile delinquency, 
experiences implementing federal programs, and promising practices that 
our team identified in our interviews with tribal officials. We then used that 
list to code the interview summaries, applying multiple codes to a 
summary if the interview covered a range of issues. This process allowed 
us to determine the number of interview summaries that contained any 
statements about each of the specific trends, practices, and experiences 
we identified throughout the third objective. Two analysts divided and 
completed an initial round of coding statements in the 12 interview 
summaries. A third analyst reviewed and validated the initial coding of 
each summary. All adjustments made by the third analyst during the 
coding validation were approved by a fourth analyst. Finally, we 


14We also conducted interviews with nontribal officials who had experience working on 
tribal juvenile justice issues to gain contextual knowledge but did not ultimately include 
them in our analysis for the third objective because they were not tribal officials. 


15We conducted site visits with federally recognized tribes and agency officials to 
California in December 2018 and to New Mexico in April 2019. 
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calculated the number of interview summaries that included any 
discussion of each specific theme we identified. 16 


In addition to these interviews, we also conducted two discussion groups 
at tribal-focused conferences with about 30 officials who volunteered to 
participate. One conference discussion group included participants from 
various fields, including child welfare, mental health, and juvenile justice 
service providers; legal professionals; advocates for children; and tribal 
leaders. The other discussion group included representatives of programs 
that were grantees of DOJ’s Coordinated Tribal Assistance Solicitation - 
Purpose Area 9: Tribal Youth Program in fiscal year 2018.17 


While we included examples from the two discussion groups we held 
during our site visits, the information we present on identified trends 
within the third objective is limited to the 12 interviews we conducted with 
tribes or tribal consortia described above. This is due to the 12 interviews 
we conducted having a similar semistructured format. 


The perspectives that the tribes, tribal consortia, and the two discussion 
groups offered are not generalizable, but they provide insights on the 
juvenile delinquency issues that some tribes encounter within their 
communities and the types of challenges they face in preventing or 
addressing these issues. 


We conducted this performance audit from August 2018 to August 2020 
in accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 


16We counted an interview as mentioning a specific issue or challenge related to 
preventing or addressing juvenile delinquency if at least one of the tribal officials 
participating in the interviews discussed it. 


17According to the fiscal year 2018 funding opportunity announcement for Coordinated 
Tribal Assistance Solicitation, the Tribal Youth Program seeks to support and enhance 
tribal efforts to prevent and reduce juvenile delinquency and strengthen a fair and 
beneficial juvenile justice system response for Native American youth. 
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American Evaluation Association. An Evaluation Roadmap for a More 
Effective Government (Washington, D.C.: Revised October 2016). 


American Evaluation Association. American Evaluation Association 
Guiding Principles for Evaluators (Washington, D.C.: July 2004). 


GAO. Critical Infrastructure Protection: Actions Needed to Address 
Significant Weaknesses in TSA’s Pipeline Security Program 
Management. GAO-19-48. Washington, D.C.: December 18, 2018. 


GAO. Military Personnel: DOD Needs to Establish Performance 
Measures for the Armed Forces Sports Program. GAO-17-542. 
Washington, D.C.: June 8, 2017. 


GAO. Foreign Assistance: Agencies Can Improve the Quality and 
Dissemination of Program Evaluations. GAO-17-316. Washington, D.C.: 
March 3, 2017. 


GAO. Foreign Assistance: Selected Agencies’ Monitoring and Evaluation 
Policies Generally Address Leading Practices. GAO-16-861R. 
Washington, D.C.: September 27, 2016. 


GAO. Federal Prison System: Justice Could Better Measure Progress 
Addressing Incarceration Challenges. GAO-15-454. Washington, D.C.: 
June 19, 2015. 


GAO. Fragmentation, Overlap, and Duplication: An Evaluation and 
Management Guide. GAO-15-49SP. Washington, D.C.: April 14, 2015. 


GAO. Managing for Results: Agencies’ Trends in the Use of Performance 
Information to Make Decisions. GAO-14-747. Washington, D.C.: 
September 26, 2014. 


GAO. Standards for Internal Control in the Federal Government. 
GAO-14-704G. Washington, D.C.: September 10, 2014. 


GAO. Defense Health Care Reform: Additional Implementation Details 
Would Increase Transparency of DOD’s Plans and Enhance 
Accountability. GAO-14-49. Washington, D.C.: November 6, 2013. 


GAO. Designing Evaluations: 2012 Revision. GAO-12-208G. Washington, 
D.C.: February 1, 2012. 
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GAO. Information Sharing: DHS Could Better Define How It Plans to Meet 
Its State and Local Mission and Improve Performance Accountability. 
GAO-11-223. Washington, D.C.: December 16, 2010. 


GAO. Employment and Training Administration: Increased Authority and 
Accountability Could Improve Research Program. GAO-10-243. 
Washington, D.C.: January 29, 2010. 


GAO. Program Evaluation: A Variety of Rigorous Methods Can Help 
Identify Effective Interventions. GAO-10-30. Washington, D.C.: November 
23, 2009. 


GAO. Managing for Results: Enhancing Agency Use of Performance 
Information for Management Decision Making. GAQ-05-927. Washington, 
D.C.: September 9, 2005. 


GAO. Tax Administration: IRS Needs to Further Refine Its Tax Filing 
Season Performance Measures. GAO-03-143. Washington, D.C.: 
November 22, 2002. 


GAO. Agency Performance Plans: Examples of Practices That Can 
Improve Usefulness to Decisionmakers. GAO/GGD/AIMD-99-69. 
Washington, D.C.: February 26, 1999. 


GAO. Results Act: An Evaluator’s Guide to Assessing Agency Annual 
Performance Plans. GGD-10.1.20. Washington, D.C.: April 1, 1998. 


GAO. Agencies’ Annual Performance Plans under the Results Act: An 
Assessment Guide to Facilitate Congressional Decisionmaking. 
GGD/AIMD-10.1.18. Washington, D.C.: February 1, 1998. 


GPRA Modernization Act of 2010, Pub. L. No. 111-352, 124 Stat. 3866 
(2011). 


Uniform Administrative Requirements, Cost Principles, and Audit 
Requirements for Federal Awards, 2 C.F.R. part 200. 
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Table 8: Selected Departments of Health and Human Services, Justice, Education, and the Interior Grant Programs That 
Grantees Could Have Used to Help Prevent or Address Tribal Juvenile Delinquency, Fiscal Years 2015-2018 


Fiscal years for which 
department issued a 
funding opportunity 

announcement or 


Discretionary awarded yearly 
Grant program Department Component office or formula? allocations to grantees 
Chafee Foster Care Program for Health and Human = Administration for Formula 2015 
Successful Transition to Services Children and Families 2016 
Adulthood 2017 
2018 
Grants for Coordination of Tribal Health and Human = Administration for Discretionary 2015 
Temporary Assistance for Services Children and Families 
Needy Families (TANF) and 
Child Welfare 
Grants to Tribes, Tribal Health and Human Administration for Discretionary 2016 
Organizations, and Migrant Services Children and Families 
Programs for Community-Based 
Child Abuse Prevention 
Programs 
National Quality Improvement Health and Human —— Administration for Discretionary 2017 
Center for Preventive Services Services Children and Families 
and Interventions in Indian 
Country 
Native American Language Health and Human —_— Administration for Discretionary 2015 
Preservation and Maintenance- Services Children and Families 2016 
Esther Martinez Immersion 2017 
2018 
Native Youth Initiative for Health and Human —— Administration for Discretionary 2016 
Leadership, Empowerment, and Services Children and Families 2017 
Development (I-LEAD) 2018 
Promoting Safe and Stable Health and Human — Administration for Formula? 2015 
Families Services Children and Families 2016 
2017 
2018 
Regional Partnership Grants to Health and Human Administration for Discretionary 2017 
Increase the Well-Being of, and Services Children and Families 
to Improve the Permanency 
Outcomes for, Children Affected 
by Substance Abuse in 
American Indian/Alaska Native 
Communities 
State and Tribal Indian Child Health and Human —— Administration for Discretionary 2016 
Welfare Act (ICWA) Services Children and Families 
Implementation Partnership 
Grants 
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Youth 
Fiscal years for which 
department issued a 
funding opportunity 
announcement or 
Discretionary awarded yearly 
Grant program Department Component office or formula? allocations to grantees 
Stephanie Tubbs Jones Child Health and Human —— Administration for Formula 2015 
Welfare Services Services Children and Families 2016 
2017 
2018 
Tribal Court Improvement Health and Human —_— Administration for Discretionary 2015 
Program Services Children and Families 2018 
Tribal Temporary Assistance for Health and Human Administration for Formula 2015 
Needy Families (Tribal TANF) = Services Children and Families 2016 
2017 
2018 
Tribal Maternal, Infant, and Health and Human Administration for Discretionary 2016 
Early Childhood Home Visiting Services Children and Families in 
Program: Development and partnership with the 
Implementation Grants Health Resources and 
Services Administration 
Tribal Maternal, Infant, and Health and Human Administration for Discretionary 2016 
Early Childhood Home Visiting Services Children and Families in 2018 
Program: Implementation and partnership with the 
Expansion Grants Health Resources and 
Services Administration 
Methamphetamine and Suicide Health and Human __ Indian Health Service Discretionary 2015 
Prevention Initiative Services 
Methamphetamine and Suicide Health and Human _ Indian Health Service Discretionary 2016 
Prevention Initiative - Services 2017 
Generation Indigenous Initiative 
Support 
Youth Regional Treatment Health and Human Indian Health Service Discretionary 2018 
Center Aftercare Pilot Project Services 
Collaborative Hubs to Reduce Health and Human ___ National Institutes of Discretionary 2017 
the Burden of Suicide among Services Health 
American Indian and Alaska 
Native Youth (U19) 
American Indian/Alaska Native Health and Human Office of Minority Health Discretionary 2017 
Health Equity Initiative Services 
Cooperative Agreements for Health and Human Substance Abuse and _ Discretionary 2015 
Project LAUNCH (Linking Services Mental Health Services 
Actions for Unmet Needs in Administration 
Children’s Health) State/Tribal 
Expansion (short title: Project 
LAUNCH Expansion Grants) 
Cooperative Agreements for Health and Human Substance Abuse and __ Discretionary 2015 


State-Sponsored Youth Suicide 


Prevention and Early 


Intervention (PPHF-2015) 


Services 


Mental Health Services 
Administration 
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Youth 
Fiscal years for which 
department issued a 
funding opportunity 
announcement or 
Discretionary awarded yearly 
Grant program Department Component office or formula? allocations to grantees 
Cooperative Agreements for Health and Human Substance Abuse and __ Discretionary 2016 
Tribal Behavioral Health (short Services Mental Health Services 2017 
title: Native Connections) Administration 2018 
Linking Actions for Unmet Health and Human = Substance Abuse and __ Discretionary 2017 
Needs in Children’s Health in Services Mental Health Services 
American Indian and Alaskan Administration 
Native Communities, U.S. 
Territories, and Pacific 
Jurisdictions Cooperative 
Agreements (short title: 
Indigenous - Project LAUNCH) 
Planning and Developing Health and Human Substance Abuse and _ Discretionary 2017 
Infrastructure to Improve the Services Mental Health Services 
Mental Health and Wellness of Administration 
Children, Youth and Families in 
American Indian/Alaska Natives 
Communities (short title: Circles 
of Care VII) 
Strategic Prevention Framework Health and Human Substance Abuse and Discretionary 2017 
- Partnerships for Success Services Mental Health Services 2018 
(short title: SPF-PFS) Administration 
Coordinated Tribal Assistance Justice Bureau of Justice Discretionary 2015 
Solicitation - Purpose Area 4: Assistance 2016 
Tribal Justice System 2017 
Infrastructure Program 2018 
Alaska Native Youth Training Justice Office of Juvenile Discretionary 2016 
and Technical Assistance Justice and Delinquency 
Project Prevention 
Coordinated Tribal Assistance Justice Office of Juvenile Discretionary 2015 
Solicitation - Purpose Area 8: Justice and Delinquency 2016 
Juvenile Tribal Healing to Prevention 2017 
Wellness Courts 2018 
Coordinated Tribal Assistance Justice Office of Juvenile Discretionary 2015 
Solicitation - Purpose Area 9: Justice and Delinquency 2016 
Tribal Youth Program Prevention 2017 
2018 
National Intertribal Youth Justice Office of Juvenile Discretionary 2017 
Leadership Development Justice and Delinquency 
Initiative? Prevention 
Supporting Tribal Youth: Justice Office of Juvenile Discretionary 2018 


Training, Technical Assistance 
and Youth Leadership 


Justice and Delinquency 
Prevention 
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Youth 
Fiscal years for which 
department issued a 
funding opportunity 
announcement or 
Discretionary awarded yearly 
Grant program Department Component office or formula? allocations to grantees 
Defending Childhood Justice Office of Juvenile Discretionary 2016 
American Indian/Alaska Native Justice and Delinquency 
Policy Initiative: Supporting Prevention 
Trauma-Informed Juvenile 
Justice Systems for Tribes 
Tribal Youth Program Training = Justice Office of Juvenile Discretionary 2015 
and Technical Assistance Justice and Delinquency 
Prevention 
Native American and Alaska Education Office of English Discretionary 2016 
Native Children in School Language Acquisition 2018 
Program 
Indian Education Demonstration Education Office of Elementary Discretionary 2015 
Grants - Native Youth and Secondary 2016 
Community Projects Education 2018 
Native American Language Education Office of Elementary Discretionary 2017 
(NAL@ED) Program and Secondary 
Education 
Indian Education Formula Education Office of Elementary Formula 2015 
Grants to Local Educational and Secondary 2016 
Agencies Education 2017 
2018 
Native Language Immersion Interior Bureau of Indian Discretionary 2017 
Cooperative Agreement Education 2018 


Source: GAO analysis of agency grant programs. | GAO-600-20 


*The Department of Health and Human Services (HHS) Promoting Safe and Stable Families grant 
program contains award funding on both a formula and discretionary basis. However, for the 
purposes of this review, we characterize it a formula grant because HHS primarily obligates program 
awards on a formula basis. 


>We identified the National Intertribal Youth Leadership Development Initiative, for which the Office of 
Juvenile Justice and Delinquency Prevention issued a solicitation inviting applicants to apply for 
funding in fiscal year 2017, as an American Indian and Alaska Native-specific discretionary grant 
program that could have helped prevent or address juvenile delinquency in tribal communities. 
However, Department of Justice officials told us that the agency received three applications but did 
not make any awards for the program. 
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DEPARTMENT OF HEALTH & HUMAN SERVICES OFFICE OF THE SECRETARY 


Assistant Secretary for Legislation 


%, 
nt Washington, DC 20201 


July 10, 2020 


Gretta L. Goodwin 
Director, Homeland Security and Justice 
U.S. Government Accountability Office 


441 G Street NW 
Washington, DC 20548 
Dear Ms. Goodwin: 


Attached are comments on the U.S. Government Accountability Office’s (GAO) report entitled, 
“Native American Youth: Agencies Incorporated Almost All Leading Practices When Assessing 


Grant Programs Addressing Delinquency” (Job Code 102972/GAO-20-600). 


The Department appreciates the opportunity to review this report prior to publication. 


Sincerely, 


Digitally signed by 
Sa ra h C * Sarah C. Arbes -S 
Date: 2020.07.10 


Arbes -S 11032-0800 


Sarah C. Arbes 
Assistant Secretary for Legislation 


Attachment 
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GENERAL COMMENTS FROM THE DEPARTMENT OF HEALTH & HUMAN 
SERVICES ON THE GOVERNMENT ACCOUNTABILITY OFFICE’S DRAFT 
REPORT ENTITLED — NATIVE AMERICAN YOUTH: AGENCIES 
INCORPORATED ALMOST ALL LEADNING PRACTICES WHEN ASSESSING 


GRANT PROGRAMS ADDRESSING DELINQUENCY (GAO-20-600) 


The U.S. Department of Health & Human Services (HHS) appreciates the opportunity from the 
Government Accountability Office (GAO) to review and comment on this draft report. 


Recommendation 1 


The Associate Commissioner of the Administration for Children and Families Children’s Bureau 
within the Department of Health and Human Services should develop a process to assess the 
reliability of grantee performance information for its Stephanie Tubbs Jones Child Welfare 
Services Program 


HHS Response 
HHS non-concurs. 


1) The Stephanie Tubbs Jones Child Welfare Services Program (title IV-B, subpart] of the 
Social Security Act) provides states and tribes broad authority for the use of funds 
consistent with statutorily identified purposes, which are focused on children and families 
involved with the child welfare system. Section 1123A of the Social Security Act 
dictates performance measurement, in part, for the title [V-B program. The performance 
measurement requirements at section 1123A apply only to state agencies and not to 
tribes. 


2) The recommendation is aimed at the leading practice for performance measurement to 
assess the reliability of grantee performance information provided in the Child and 
Family Services Plan (CFSP). However the primary statutorily mandated performance 
measurement requirements related to title [V-B, subpart 1 do not apply to tribes. 


3) The CFSP tribes submit in order to access funding under title [V-B, subpart | is primarily 
used to describe how tribes will implement the title IV-B program. 


Federal regulations provide that a participating state or Indian Tribe must base the 
development of the CFSP on a planning process that includes involvement and 
consultation with a wide range of stakeholders, including relevant state and tribal 
agencies, community-based service providers, and parents and youth who are 
involved or have experience with the child welfare system. The plan must include 
goals, objective and measures of progress that are documented in the plan and 
addressed in consultation with community stakeholders. 


4) The CFSP and the annual update that tribes submit to ACF in order to receive funding 


under title IV-B, subpart 1 is also required to be developed jointly with ACF in 
accordance with section 422(a) of the Social Security Act. 
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GENERAL COMMENTS FROM THE DEPARTMENT OF HEALTH & HUMAN 
SERVICES ON THE GOVERNMENT ACCOUNTABILITY OFFICE’S DRAFT 
REPORT ENTITLED — NATIVE AMERICAN YOUTH: AGENCIES 
INCORPORATED ALMOST ALL LEADNING PRACTICES WHEN ASSESSING 


GRANT PROGRAMS ADDRESSING DELINQUENCY (GAO-20-600) 


As defined in federal regulations at 45 CFR 1357.10(c), “Joint planning involves 
discussions, consultation, and negotiation between ACF and the State or Indian 
Tribe in all areas of CFSP creation such as, but not limited to, identifying the 
service needs of children, youth, and families; selecting the unmet service needs 
that will be addressed; developing goals and objectives that will result in 
improving outcomes for children and families; developing a plan to meet the 
matching requirements; and establishing a more comprehensive, coordinated and 
effective child and family services delivery system. The expectation of joint 
planning is that both ACF and the State or Indian Tribe will reach agreement on 
substantive and procedural matters related to the CFSP.” 


GAO recommended that ACF validate the efficacy of a plan that we participated in 
developing. Any additional validation beyond joint planning, existing requirements for 
consultation and coordination would be overly burdensome, and resource intensive. 
Further, on page 21 of this report, GAO provides an example on how to assess data 
quality — including inquiring about how data are collected or asking tribal officials’ 
opinions on the quality of the data. These are essentially the very types of questions that 
are addressed in joint planning and the development of the CFSP. Therefore, existing 
procedures are appropriate and sufficient and development of further data validation 
procedures is unnecessary. 
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United States Department of the Interior 


OFFICE OF THE SECRETARY 
Washington, DC 20240 


JUL 21 2020 


Ms. Gretta Goodwin 

Director, Homeland Security and Justice 
U.S. Government Accountability Office 
441 G Street, N.W. 

Washington, DC 20548 


Dear Ms. Goodwin: 


Thank you for providing the Department of the Interior (Department) the opportunity to review and comment 
on the draft Government Accountability Office (GAO) report entitled, Native American Youth: Agencies 
Incorporated Almost All Leading Practices When Assessing Grant Programs Addressing Delinquency (GAO- 
20-600). We appreciate GAO’s evaluation of the Bureau of Indian Education’s (BIE) Native Language 
Immersion Cooperative Agreement program. 


The GAO issued two recommendations to the Department as part of its overall findings to improve the 
administration and oversight of the BIE native language grant program. Below is a summary of actions taken or 
planned to implement the recommendations. 


Recommendation 2: The Director of the Department of the Interior’s Bureau of Indian Education 
should take steps to alert grantees of the Native Language Immersion Cooperative Agreement when 
they are late in submitting performance reports. 


Response: Concur. As highlighted within the report, funding for the Native Language Immersion Cooperative 
Agreement was unanticipated by the BIE. In response to receiving this unanticipated funding the BIE has 
tasked its Sovereignty in Indian Education (SIE) Office with designing and implementing the Native Language 
Immersion Cooperative Agreement program. However, as highlighted in the report, the BIE in general, and the 
SIE Office specifically, faces human capital challenges. Nonetheless, as the Native Language Immersion 
Cooperative Agreement program matures and the SIE Office continues to fill critical positions, the BIE 
anticipates that the issue of timely and regular performance reporting will be addressed. 


Recommendation 3: The Director of the Department of the Interior’s Bureau of Indian Education 
should develop a process to assess the quality and reliability of a sample of grantee performance data 
for the Native Language Immersion Cooperative Agreement. 


Response: Concur. As highlighted within the report, the BIE recently established a Performance Office. The 
Performance Office will be tasked with assisting the SIE Office in inventorying, reviewing, and improving data 
collection and analysis. As part of this work, the BIE anticipates that the Native Language Immersion 
Cooperative Agreement program will develop a process to assess the quality and reliability of data. 


If you have any questions, or need additional information, please contact Mr. Travis Clark, Chief Performance 
Officer, BIE, at (202) 208-3612 or via email: travis.clark@bie.edu. 


Sincerely, 


Tara Sweeney 
Assistant Secretary — Indian Affairs 
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UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE OF ELEMENTARY AND SECONDARY EDUCATION 


Government Accountability Office 
Gretta L. Goodwin 

Director, Homeland Security and Justice 
441 GSt. NW 

Washington, D.C. 20548 


Dear Ms. Goodwin: 


Thank you for providing the U.S. Department of Education (“Department”) with the opportunity 
to respond to the draft Government Accountability Office (GAO) report, regarding Native 
American Youth: Agencies Incorporating Almost All Leading Practices When Assessing Grant 
Programs Addressing Delinquency (GAO Report 20-600). While there are no recommendations 
for the Department in the draft GAO report, we would like to take this opportunity to provide 
two comments pertaining to the draft report. We include additional technical comments in the 
enclosure. 


First, we wish to reemphasize a concern expressed by Department staff during this GAO 
engagement; we have noted throughout the engagement and note again in response to the draft 
report, that the use of the term “delinquency” has pejorative implications, and does not appear 
appropriate when referencing programs that are not intended to address delinquency. Native 
American students who have been involved with the justice system for various reasons should 
not be automatically assumed to be and labelled as “juvenile delinquents.” 


Further, we disagree with the use of the phrase “Grant Programs Addressing Delinquency” in the 
title of the report because it implies that the purpose of each of the programs listed in the report, 
including the programs funded by this Department, is to address the topic of “delinquency.” This 
is not the case with the Department programs and for the Department’s mission generally, which 
is to improve educational opportunities, access and equity for all children, including Native 
American students. For the Department’s programs listed in the report, Native American students 
are eligible participants for projects designed to improve educational outcomes and/or teach 
Native languages, and generally there is no connection with justice systems. The Department 
does not view the students served by the grantees as “potential delinquents.” We request that 
GAO consider inserting a statement in the draft GAO report that explains that while the 
Department of Education programs cited in the report “could” potentially reduce student 
interactions with the justice system, that is not the primary focus or statutory purpose of these 
programs. 


On a related note, the draft GAO report identifies some promising practices to reduce 
involvement of Native youth with justice systems. However, these practices are only listed by 
those Tribes that were interviewed; there is no mention of research or evidence regarding these 
or other potential promising best practices. It would strengthen the draft GAO report to include 
research that supports some of these interventions, where applicable. For example, various 
jurisdictions are adopting the restorative justice model used by some Tribes due to its success. It 
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would be helpful to include mention of this and other best practice successes in the draft GAO 
report and any related research. 
We have attached technical comments on the report which we hope will be helpful. Again, we 
appreciate the opportunity to review and provide comments on this draft GAO report. If you 
have any questions, please contact our Audit Liaison Officer Jed Sorokin-Altmann, jed.sorokin- 
altmann@ed.gov. 

Sincerely, 


Digitally signed by Ruth 


Ruth Ryder Sis 2020.07.08 


15:38:42 -04'00' 
Ruth Ryder 
Deputy Assistant Secretary for Formula Grants 
U.S. Department of Education, Office of 
Elementary and Secondary Education 


Enclosure 
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’ ‘oat The Government Accountability Office, the audit, evaluation, and investigative 

GAO Ss Mission arm of Congress, exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability of the 
federal government for the American people. GAO examines the use of public 
funds; evaluates federal programs and policies; and provides analyses, 
recommendations, and other assistance to help Congress make informed 
oversight, policy, and funding decisions. GAO’s commitment to good government 
is reflected in its core values of accountability, integrity, and reliability. 
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Obtaining Copies of through our website. Each weekday afternoon, GAO posts on its website newly 
GAO Reports and released reports, testimony, and correspondence. You can also subscribe to 
. GAO’s email updates to receive notification of newly posted products. 
Testimony 


Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and 
distribution and depends on the number of pages in the publication and whether 
the publication is printed in color or black and white. Pricing and ordering 
information is posted on GAO’s website, https://www.gao.gov/ordering.htm. 


Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 
TDD (202) 512-2537. 


Orders may be paid for using American Express, Discover Card, MasterCard, 
Visa, check, or money order. Call for additional information. 


. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 
Connect with GAO Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. 
Visit GAO on the web at https://www.gao.gov. 
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: Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, 
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